[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2019015477A1 - Image correction method, computer readable storage medium and computer device - Google Patents

Image correction method, computer readable storage medium and computer device Download PDF

Info

Publication number
WO2019015477A1
WO2019015477A1 PCT/CN2018/094471 CN2018094471W WO2019015477A1 WO 2019015477 A1 WO2019015477 A1 WO 2019015477A1 CN 2018094471 W CN2018094471 W CN 2018094471W WO 2019015477 A1 WO2019015477 A1 WO 2019015477A1
Authority
WO
WIPO (PCT)
Prior art keywords
deformation
face
profile
image
operator
Prior art date
Application number
PCT/CN2018/094471
Other languages
French (fr)
Chinese (zh)
Inventor
曾元清
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2019015477A1 publication Critical patent/WO2019015477A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/24Aligning, centring, orientation detection or correction of the image
    • G06V10/247Aligning, centring, orientation detection or correction of the image by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids

Definitions

  • the present application relates to the field of image processing, and in particular to an image correction method, a non-transitory computer readable storage medium, and a computer device.
  • Photographing has gradually become a part of people's lives, and people can take pictures or portraits with mobile devices with cameras anytime, anywhere.
  • the number of people with visual impairment is very large, and the deformation caused by the lens during photographing affects the imaging effect of the photographed face.
  • an image correction method a non-transitory computer readable storage medium, and a computer device are provided.
  • An image correction method comprising:
  • the deformation profile is subjected to deformation processing using the updated deformation operator.
  • One or more non-transitory computer readable storage media containing computer executable instructions that, when executed by one or more processors, cause the processor to:
  • the deformation profile is subjected to deformation processing using the updated deformation operator.
  • a computer device comprising a memory and a processor, the memory storing computer readable instructions, wherein when executed by the processor, the processor causes the processor to:
  • the deformation profile is subjected to deformation processing using the updated deformation operator.
  • the image correcting method, the non-volatile computer readable storage medium, and the computer device in the embodiments of the present application improve the imaging effect of the face.
  • FIG. 1 is a diagram showing the internal structure of a computer device in an embodiment
  • FIG. 2 is a flow chart of an image correction method in one embodiment
  • FIG. 3 is a flow chart of an image correction method in another embodiment
  • FIG. 4 is a schematic diagram showing deformation of a photo taken by a user wearing glasses in an embodiment
  • FIG. 5 is a schematic diagram showing a fitting curve obtained by fitting a deformation region in the contour of the face in FIG. 4 in one embodiment
  • Figure 6 is an internal block diagram of an image correcting device in one embodiment
  • Figure 7 is a schematic illustration of an image processing circuit in one embodiment.
  • first may be referred to as a second client
  • second client may be referred to as a first client, without departing from the scope of the present application.
  • Both the first client and the second client are clients, but they are not the same client.
  • FIG. 1 is a schematic diagram showing the internal structure of a computer device in an embodiment.
  • the computer device includes a processor coupled through a system bus, a non-volatile storage medium, an internal memory, a network interface, a display screen, and an input device.
  • the non-volatile storage medium of the computer device stores an operating system and computer readable instructions.
  • the computer readable instructions are executed by the processor to implement an image correction method.
  • the processor is used to provide computing and control capabilities to support the operation of the entire computer device.
  • the internal memory in the computer device provides an environment for the operation of computer executable instructions in a non-volatile storage medium.
  • the network interface is used for network communication with servers or other devices.
  • the display screen of the computer device may be a liquid crystal display or an electronic ink display screen.
  • the input device may be a touch layer covered on the display screen, or may be a button, a trackball or a touchpad provided on a computer device casing, or may be An external keyboard, trackpad, or mouse.
  • the computer device can be a cell phone, a tablet or a personal digital assistant or a wearable device or the like. It will be understood by those skilled in the art that the structure shown in FIG. 1 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied.
  • the specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
  • FIG. 2 is a flow chart of an image correction method in one embodiment. As shown in FIG. 2, an image correction method is run on a computer device, and the method may include the operation shown in FIG. 2.
  • Operation 202 detecting a deformation profile of the image including the face.
  • the image can be taken by an electronic device with a camera.
  • the image contains a human face.
  • the image can be an image stored in an album or on a network.
  • the deformation profile refers to a facial contour formed by deformation of a facial contour caused by refraction or the like of the face.
  • Refraction refers to the refraction of the area of the lens due to myopia or farsighted lenses.
  • the image can be detected by a machine learning model to obtain a deformation profile containing the face in the image.
  • the computer device needs to collect normal facial contour samples and facial contour samples containing deformation contours as training samples for machine learning, and train machine learning models through training samples to obtain machine learning of facial contours. model.
  • the computer device can also recognize facial features in the image through a key feature point extraction algorithm for the face.
  • the facial features may include several features such as the eyes, mouth, nose, and eyebrows.
  • the face key feature points may include 2 eyeball center points, 4 eye corner points, 2 nozzle midpoints, and 2 mouth corner points.
  • the computer device can use the susan operator to extract the edge and corner features of the local region.
  • the principle of the Susan operator is to use a circular area with a radius of pixels as a mask to examine how the pixel values of all points in the region of the image coincide with the values of the current point.
  • the computer device can also detect the facial contour and the deformation contour of the face by using an edge detection operator such as sobel or canny.
  • a deformation trend of the deformation profile is determined, and a corresponding deformation operator is selected according to the deformation trend.
  • the shape of the deformation profile can be obtained.
  • the computer device compares the shape of the deformed contour with the shape of the reference facial contour, and whether the deformation trend of the deformed contour is reduced or expanded.
  • the reference face contour refers to a preset face contour as a standard.
  • the operation 204 includes: when determining that the deformation trend of the deformation profile is reduced, selecting a first deformation operator; and when determining that the deformation trend of the deformation profile is expanding, selecting a second deformation calculation child.
  • the computer device may pre-establish a correspondence relationship between the deformation trend and the deformation operator, and after detecting the deformation trend, obtain a corresponding deformation operator from the corresponding relationship between the deformation trend and the deformation operator according to the deformation trend.
  • a deformation operator is a parameter that performs a deformation operation on an image.
  • Operation 206 identifying a facial contour of the face in the image.
  • the computer device may employ a machine learning model to identify facial contours in the image.
  • the machine learning model is obtained by training the training samples in advance, or by extracting key feature points of the face.
  • Operation 208 curve fitting the deformed contour of the face according to the facial contour to obtain a fitting curve.
  • the computer device may fit the corresponding reference facial contour according to the remaining contours remaining in the facial contour except the deformed contour.
  • the computer device obtains a curve contoured face contour, that is, a fitting curve, according to the reference face contour and the detected facial contour in the image.
  • Curve fitting can use scatter points other than the deformed contour in the contour of the face, select the appropriate curve type for variable transformation, make the two variables after the transformation have a linear relationship, and find the linear equation and variance according to the least squares method, and straighten the line.
  • the equation is converted to a function expression about the original variable.
  • the deformation operator is adjusted according to the fitting curve and the deformation profile to obtain an updated deformation operator.
  • the computer device adjusts the deformation operator according to the difference between the fitting curve and the deformation profile to obtain the updated deformation operator.
  • the deformation operator can be an affine transformation matrix. Each affine transform corresponds to a multiplication of a rectangle and a vector. Affine transformations can be achieved through a series of atomic transformations, including translation, scaling, flipping, rotation, and miscutting.
  • the simulation transformation is represented by a 3 ⁇ 3 matrix, the last of which is (0, 0, 1).
  • the transformation matrix transforms the original coordinates (x 1 , y 1 ) into new coordinates (x 2 , y 2 ).
  • the original coordinates and the new coordinates are adjacent to the three-dimensional column of the last behavior (1), and the original column vector is multiplied by the transformation.
  • the matrix gets a new column vector, as in equation (1).
  • the deformation profile can be obtained by translational transformation.
  • the transformation matrix of the translation transformation can be
  • Operation 212 deforming the deformation profile by using the updated deformation operator.
  • the computer device deforms the deformation profile by the updated deformation operator to obtain the corrected contour.
  • the image correction method in the embodiment of the present application detects the deformation profile of the face in the image, selects a corresponding deformation operator according to the deformation trend of the deformation profile, detects the contour of the face, and formulates the deformation profile according to the contour of the face.
  • the fitting curve is obtained, and the deformation operator is adjusted according to the fitting curve and the deformation contour to obtain the updated deformation operator, and the deformed contour can be corrected according to the updated deformation operator to obtain the corrected facial contour and improved.
  • the imaging effect of the face is obtained, and the deformation operator is adjusted according to the fitting curve and the deformation contour to obtain the updated deformation operator, and the deformed contour can be corrected according to the updated deformation operator to obtain the corrected facial contour and improved.
  • identifying the facial contour in the image may precede the operation 202.
  • including the deformation profile of the face in the detection image comprises: acquiring the deformation profile including the face according to a color of the skin.
  • face detection based on skin color may include pre-processing, skin color segmentation based on skin color model; connected domain analysis, face region localization.
  • the preprocessing can use Gaussian filtering and histogram equalization.
  • the skin color model can adopt a color model of the YCbCr space, where Y refers to luminance information, and Cb and Cr are chrominance information.
  • the computer device establishes a Gaussian model of the skin color according to the mean and variance of the skin color, and obtains a face probability map through the Gaussian model of the skin color, and uses the binarization to obtain the face color binary image.
  • the computer device can perform connected domain analysis on the input image to obtain a minimum circumscribed rectangle of the binary image, that is, a face region.
  • the computer device first displays the pixels in the binary image that meet the preset connection rule by the same label, obtains the connected area contour of the binary image, and obtains the minimum circumscribed rectangle of the connected area.
  • Methods for connecting domain tags include pixel notation, line notation, and region growing.
  • the deforming contour of the detected image includes: determining whether there is glasses in the image, and when there is a human face in the image, detecting whether the area where the glasses is located includes a face, and when the glasses are located When the face is included, the facial contour of the area where the glasses are located is acquired, and the facial contour of the area where the glasses are located is used as the deformed contour including the face.
  • an image correction method includes:
  • operation 302 it is determined whether there is glasses in the image. When there are glasses in the image, operation 304 is performed, and when there is no glasses in the image, the process ends.
  • operation 304 it is detected whether the area where the glasses are located includes a facial contour.
  • operation 306 is performed, and when the area where the glasses is located does not include a facial contour, the processing ends.
  • operation 306 it is determined whether the deformation trend of the facial contour of the region where the glasses are located is reduced. When the deformation trend is reduced, operation 308 is performed, and when the deformation trend is not reduced, operation 310 is performed.
  • Operation 308 selecting the first deformation operator, and performing operation 312.
  • the first deformation operator is a myopia deformation operator.
  • Operation 310 selecting the second deformation operator, performs operation 312.
  • the second deformation operator is a far vision mirror deformation operator.
  • Operation 312 identifying a facial contour of the face in the image.
  • Operation 316 adjusting the deformation operator according to the fitting curve and the deformation profile to obtain an updated deformation operator.
  • Operation 318 deforming the deformation profile by using the updated deformation operator.
  • the corresponding deformation operator is selected according to the deformation trend of the facial contour in the region where the glasses are located, and the facial contour is detected according to the face.
  • the contour of the contour is fitted to the deformation profile to obtain a fitting curve.
  • the deformation operator is adjusted according to the fitting curve and the deformation contour to obtain the updated deformation operator.
  • the deformation deformation can be corrected according to the updated deformation operator.
  • the contour of the back face improves the imaging effect of the face, so that the user wearing the glasses gets a better portrait photo when shooting. When the area where the eye is located does not contain the face area, it ends, which reduces data processing.
  • FIG. 4 is a schematic diagram showing deformation of a photo taken by a user wearing glasses in an embodiment.
  • FIG. 4 the facial contour of the area in which the glasses are located is inwardly concave due to the refraction of the glasses, and if there is a fault in both the facial contour 42 and the facial contour 44, the facial contour 44 represents the deformed contour.
  • FIG. 5 is a schematic diagram showing a fitting curve obtained by fitting a deformation region in the contour of the face in FIG. 4 in one embodiment.
  • a fitted curve 46 is obtained by fitting a fault region between the facial contour 42 and the facial contour 44.
  • the selected deformation operator can be adjusted according to the fitting curve 46 and the face contour 44 to obtain an updated deformation operator.
  • the facial contour 44 is deformed according to the updated deformation operator to obtain a corrected facial contour.
  • the above image correction method can be applied to a photo editor.
  • the image correction method is used to correct the photo in the photo editor.
  • the operations in the flowchart of the method of the embodiment of the present application are sequentially displayed in accordance with the indication of the arrows, but the operations are not necessarily performed in the order indicated by the arrows. Except as explicitly stated herein, the execution of these operations is not strictly limited, and may be performed in other sequences. Moreover, at least a part of the operations in the method flowchart of the embodiment of the present application may include multiple sub-operations or multiple stages, which are not necessarily performed at the same time, but may be executed at different times. The order of execution is not necessarily performed sequentially, but may be performed alternately or alternately with at least a portion of the sub-operations or phases of other operations or other operations.
  • FIG. 6 is an internal block diagram of an image correcting device in one embodiment.
  • an image correction device 600 includes a detection module 602 , a selection module 604 , an identification module 606 , a fitting module 608 , an adjustment module 610 , and a correction module 612 . among them:
  • the detection module 602 detects a deformation profile that includes a face in the image.
  • the selecting module 604 is configured to determine a deformation trend of the deformation profile, and select a corresponding deformation operator according to the deformation trend.
  • the identification module 606 is for identifying a facial contour of the face in the image.
  • the fitting module 608 is configured to perform curve fitting on the deformation profile of the face according to the facial contour to obtain a fitting curve.
  • the adjustment module 610 is configured to adjust the deformation operator according to the fitting curve and the deformation profile to obtain an updated deformation operator.
  • the correction module 612 is configured to deform the deformation profile by using the updated deformation operator.
  • the image correcting device in the embodiment of the present application detects the deformation profile of the face in the image, selects a corresponding deformation operator according to the deformation trend of the deformation profile, detects the contour of the face, and formulates the deformation profile according to the contour of the face.
  • the fitting curve is obtained, and the deformation operator is adjusted according to the fitting curve and the deformation contour to obtain the updated deformation operator, and the deformed contour can be corrected according to the updated deformation operator to obtain the corrected facial contour and improved.
  • the imaging effect of the face is obtained, and the deformation operator is adjusted according to the fitting curve and the deformation contour to obtain the updated deformation operator, and the deformed contour can be corrected according to the updated deformation operator to obtain the corrected facial contour and improved.
  • the detection module 602 is further configured to acquire the deformation profile including the face according to the color of the skin.
  • the detecting module 602 is further configured to determine whether there is glasses in the image, and when there is glasses in the image, detecting whether the area where the glasses is located includes a face, and when the area where the glasses is located includes a face, acquiring the The facial contour of the region where the glasses are located, and the facial contour of the region where the glasses are located is used as the deformed contour including the face.
  • the selecting module 604 is further configured to: when determining that the deformation trend of the deformation contour is reduced, selecting a first deformation operator; and when determining that the deformation trend of the deformation contour is expanding, selecting the second Deformation operator.
  • the detection module 602 is further configured to identify a deformation profile of the image that includes the face using a machine learning model.
  • each module in the image correcting device described above is for illustrative purposes only. In other embodiments, the image correcting device may be divided into different modules as needed to perform all or part of the functions of the image correcting device.
  • the various modules in the image correcting device described above may be implemented in whole or in part by software, hardware, and combinations thereof.
  • the above modules may be embedded in the hardware in the processor or in the memory in the server, or may be stored in the memory in the server, so that the processor calls the corresponding operations of the above modules.
  • the terms "component”, “module” and “system” and the like are intended to mean a computer-related entity, which may be hardware, a combination of hardware and software, software, or software in execution.
  • a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer.
  • both the application running on the server and the server can be components.
  • One or more components can reside within a process and/or executed thread, and the components can be located within one computer and/or distributed between two or more computers.
  • the embodiment of the present application also provides a non-transitory computer readable storage medium.
  • One or more non-transitory computer readable storage media containing computer executable instructions that, when executed by one or more processors, cause the processor to perform an image correction method as described above.
  • the embodiment of the present application further provides a computer device.
  • the above computer device includes an image processing circuit, and the image processing circuit may be implemented by hardware and/or software components, and may include various processing units defining an ISP (Image Signal Processing) pipeline.
  • Figure 7 is a schematic illustration of an image processing circuit in one embodiment. As shown in FIG. 7, for convenience of explanation, only various aspects of the image processing technique related to the embodiment of the present application are shown.
  • the image processing circuit includes an ISP processor 740 and a control logic 750.
  • the image data captured by imaging device 710 is first processed by ISP processor 740, which analyzes the image data to capture image statistical information that may be used to determine and/or control one or more control parameters of imaging device 710.
  • Imaging device 710 can include a camera having one or more lenses 712 and image sensors 714.
  • Image sensor 714 can include a color filter array (such as a Bayer filter) that can capture light intensity and wavelength information captured with each imaging pixel of image sensor 714 and provide a set of primitives that can be processed by ISP processor 740 Image data.
  • Sensor 720 can provide raw image data to ISP processor 740 based on sensor 720 interface type.
  • the sensor 720 interface may utilize a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above.
  • SMIA Serial Mobile Imaging Architecture
  • the ISP processor 740 processes the raw image data pixel by pixel in a variety of formats.
  • each image pixel can have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 740 can perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Among them, image processing operations can be performed with the same or different bit depth precision.
  • ISP processor 740 can also receive pixel data from image memory 730. For example, raw pixel data is sent from the sensor 720 interface to image memory 730, which is then provided to ISP processor 740 for processing.
  • Image memory 730 can be part of a memory device, a storage device, or a separate dedicated memory within an electronic device, and can include DMA (Direct Memory Access) features.
  • DMA Direct Memory Access
  • ISP processor 740 can perform one or more image processing operations, such as time domain filtering.
  • the processed image data can be sent to image memory 730 for additional processing before being displayed.
  • the "front end” processing data may also be received directly from the ISP processor 740, or the "front end” processing data may be received from the image memory 730, and the "front end” processing data may be processed in the original domain and in the RGB and YCbCr color spaces.
  • the processed image data can be output to display 770 for viewing by a user and/or further processed by a graphics engine or GPU (Graphics Processing Unit).
  • graphics engine or GPU Graphics Processing Unit
  • the output of ISP processor 740 can also be sent to image memory 730, and display 770 can read image data from image memory 730.
  • image memory 730 can be configured to implement one or more frame buffers.
  • the output of ISP processor 740 can be sent to encoder/decoder 760 to encode/decode image data. The encoded image data can be saved and decompressed before being displayed on the display 770 device.
  • the ISP processor 740 processes the image data by performing VFE (Video Front End) processing and CPP (Camera Post Processing) processing on the image data.
  • VFE processing of the image data may include correcting the contrast or brightness of the image data, modifying the digitally recorded illumination state data, performing compensation processing on the image data (such as white balance, automatic gain control, gamma correction, etc.), and performing image data.
  • CPP processing of image data may include scaling the image, providing a preview frame and a recording frame to each path. Among them, CPP can use different codecs to process preview frames and record frames.
  • the image data processed by the ISP processor 740 can be sent to the beauty module 760 for aesthetic processing of the image prior to being displayed.
  • the beauty treatment of the image data by the beauty module 760 may include: whitening, freckle, dermabrasion, face-lifting, acne, eye enlargement, and the like.
  • the beauty module 760 can be a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) in a computer device.
  • the processed data of the beauty module 760 can be sent to the encoder/decoder 770 to encode/decode the image data.
  • the encoded image data can be saved and decompressed before being displayed on the display 780 device.
  • the statistics determined by the ISP processor 740 can be sent to the control logic 750 unit.
  • the statistics may include image sensor 714 statistics such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, lens 712 shading correction, and the like.
  • Control logic 750 can include a processor and/or a microcontroller that executes one or more routines, such as firmware, and one or more routines can determine control parameters and control of imaging device 710 based on received statistical data.
  • the control parameters may include sensor 720 control parameters (eg, gain, integration time for exposure control), camera flash control parameters, lens 712 control parameters (eg, focus or zoom focal length), or a combination of these parameters.
  • the ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), and lens 712 shading correction parameters.
  • the image correction method described above is implemented by a processor in the image processing technique of FIG.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

An image correction method comprises: detecting a deformed outline containing a face in an image; determining a deformation trend of the deformed outline, and selecting a deformation operator corresponding to the deformation trend; identifying a facial outline of the face in the image; performing a curve fitting on the deformed outline of the face according to the facial outline to obtain a fitted curve; adjusting the deformation operator according to the fitted curve and the deformed outline to obtain an updated deformation operator; and performing a deformation processing on the deformed outline using the updated deformation operator.

Description

图像矫正方法、计算机可读存储介质和计算机设备Image correction method, computer readable storage medium, and computer device
相关申请的交叉引用Cross-reference to related applications
本申请要求于2017年07月18日提交中国专利局、申请号为2017105878139、发明名称为“图像矫正方法、装置、计算机可读存储介质和计算机设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。The present application claims the priority of the Chinese Patent Application entitled "Image Correction Method, Apparatus, Computer Readable Storage Medium, and Computer Equipment", filed on July 18, 2017, the Chinese Patent Office, Application No. 2017105878139, the entire contents of which is incorporated herein by reference. This is incorporated herein by reference.
技术领域Technical field
本申请涉及图像处理领域,特别是涉及一种图像矫正方法、非易失性计算机可读存储介质和计算机设备。The present application relates to the field of image processing, and in particular to an image correction method, a non-transitory computer readable storage medium, and a computer device.
背景技术Background technique
拍照已逐步成为人们生活中的一部分,人们可随时随地利用带摄像头的移动设备拍摄风景或人像等。而视力缺陷的人群数量十分庞大,拍照时由于镜片导致的变形影响了拍摄的面部成像效果。Photographing has gradually become a part of people's lives, and people can take pictures or portraits with mobile devices with cameras anytime, anywhere. The number of people with visual impairment is very large, and the deformation caused by the lens during photographing affects the imaging effect of the photographed face.
发明内容Summary of the invention
根据本申请的各种实施例,提供一种图像矫正方法、非易失性计算机可读存储介质和计算机设备。According to various embodiments of the present application, an image correction method, a non-transitory computer readable storage medium, and a computer device are provided.
一种图像矫正方法,包括:An image correction method comprising:
检测图像中包含脸部的形变轮廓;Detecting a deformation profile of the face in the image;
判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子;Determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend;
识别所述图像中所述脸部的脸部轮廓;Identifying a facial contour of the face in the image;
根据所述脸部轮廓对所述脸部的形变轮廓进行曲线拟合得到拟合曲线;Performing a curve fitting on the deformation profile of the face according to the facial contour to obtain a fitting curve;
根据所述拟合曲线及所述形变轮廓对所述形变算子进行调整得到更新后 的形变算子;及Adjusting the deformation operator according to the fitting curve and the deformation profile to obtain an updated deformation operator; and
采用所述更新后的形变算子对所述形变轮廓进行形变处理。The deformation profile is subjected to deformation processing using the updated deformation operator.
一个或多个包含计算机可执行指令的非易失性计算机可读存储介质,当所述计算机可执行指令被一个或多个处理器执行时,使得所述处理器执行以下操作:One or more non-transitory computer readable storage media containing computer executable instructions that, when executed by one or more processors, cause the processor to:
检测图像中包含脸部的形变轮廓;Detecting a deformation profile of the face in the image;
判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子;Determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend;
识别所述图像中所述脸部的脸部轮廓;Identifying a facial contour of the face in the image;
根据所述脸部轮廓对所述脸部的形变轮廓进行曲线拟合得到拟合曲线;Performing a curve fitting on the deformation profile of the face according to the facial contour to obtain a fitting curve;
根据所述拟合曲线及所述形变轮廓对所述形变算子进行调整得到更新后的形变算子;及Adjusting the deformation operator according to the fitting curve and the deformation profile to obtain an updated deformation operator; and
采用所述更新后的形变算子对所述形变轮廓进行形变处理。The deformation profile is subjected to deformation processing using the updated deformation operator.
一种计算机设备,包括存储器及处理器,所述存储器中储存有计算机可读指令,所述指令被所述处理器执行时,使得所述处理器执行以下操作:A computer device comprising a memory and a processor, the memory storing computer readable instructions, wherein when executed by the processor, the processor causes the processor to:
检测图像中包含脸部的形变轮廓;Detecting a deformation profile of the face in the image;
判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子;Determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend;
识别所述图像中所述脸部的脸部轮廓;Identifying a facial contour of the face in the image;
根据所述脸部轮廓对所述脸部的形变轮廓进行曲线拟合得到拟合曲线;Performing a curve fitting on the deformation profile of the face according to the facial contour to obtain a fitting curve;
根据所述拟合曲线及所述形变轮廓对所述形变算子进行调整得到更新后的形变算子;及Adjusting the deformation operator according to the fitting curve and the deformation profile to obtain an updated deformation operator; and
采用所述更新后的形变算子对所述形变轮廓进行形变处理。The deformation profile is subjected to deformation processing using the updated deformation operator.
本申请实施例中的图像矫正方法、非易失性计算机可读存储介质和计算机设备,提高了面部的成像效果。The image correcting method, the non-volatile computer readable storage medium, and the computer device in the embodiments of the present application improve the imaging effect of the face.
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。Details of one or more embodiments of the present application are set forth in the accompanying drawings and description below. Other features, objects, and advantages of the invention will be apparent from the description and appended claims.
附图说明DRAWINGS
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative work for those skilled in the art.
图1为一个实施例中计算机设备的内部结构图;1 is a diagram showing the internal structure of a computer device in an embodiment;
图2为一个实施例中图像矫正方法的流程图;2 is a flow chart of an image correction method in one embodiment;
图3为另一个实施例中图像矫正方法的流程图;3 is a flow chart of an image correction method in another embodiment;
图4为一个实施例中戴眼镜用户拍照发生形变的示意图;FIG. 4 is a schematic diagram showing deformation of a photo taken by a user wearing glasses in an embodiment; FIG.
图5为一个实施例中对图4中的脸部轮廓中形变区域进行拟合得到拟合曲线的示意图;FIG. 5 is a schematic diagram showing a fitting curve obtained by fitting a deformation region in the contour of the face in FIG. 4 in one embodiment; FIG.
图6为一个实施例中图像矫正装置的内部框图;Figure 6 is an internal block diagram of an image correcting device in one embodiment;
图7为一个实施例中图像处理电路的示意图。Figure 7 is a schematic illustration of an image processing circuit in one embodiment.
具体实施方式Detailed ways
为了使本申请的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本申请进行进一步详细说明。应当理解,此处所描述的具体实施例仅仅用以解释本申请,并不用于限定本申请。In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting.
可以理解,本申请所使用的术语“第一”、“第二”等可在本文中用于描述各种元件,但这些元件不受这些术语限制。这些术语仅用于将第一个元件与另一个元件区分。举例来说,在不脱离本申请的范围的情况下,可以将第一客户端称为第二客户端,且类似地,可将第二客户端称为第一客户端。第一客户端和第二客户端两者都是客户端,但其不是同一客户端。It will be understood that the terms "first", "second" and the like, as used herein, may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, a first client may be referred to as a second client, and similarly, a second client may be referred to as a first client, without departing from the scope of the present application. Both the first client and the second client are clients, but they are not the same client.
图1为一个实施例中计算机设备的内部结构示意图。如图1所示,该计算机设备包括通过系统总线连接的处理器、非易失性存储介质、内存储器、网络接口、显示屏和输入装置。其中,计算机设备的非易失性存储介质存储有 操作系统和计算机可读指令。该计算机可读指令被处理器执行时以实现一种图像矫正方法。该处理器用于提供计算和控制能力,支撑整个计算机设备的运行。计算机设备中的内存储器为非易失性存储介质中的计算机可执行指令的运行提供环境。网络接口用于与服务器或其他设备进行网络通信。计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏等,输入装置可以是显示屏上覆盖的触摸层,也可以是计算机设备外壳上设置的按键、轨迹球或触控板,也可以是外接的键盘、触控板或鼠标等。该计算机设备可以是手机、平板电脑或者个人数字助理或穿戴式设备等。本领域技术人员可以理解,图1中示出的结构,仅仅是与本申请方案相关的部分结构的框图,并不构成对本申请方案所应用于其上的计算机设备的限定,具体的计算机设备可以包括比图中所示更多或更少的部件,或者组合某些部件,或者具有不同的部件布置。FIG. 1 is a schematic diagram showing the internal structure of a computer device in an embodiment. As shown in FIG. 1, the computer device includes a processor coupled through a system bus, a non-volatile storage medium, an internal memory, a network interface, a display screen, and an input device. Wherein, the non-volatile storage medium of the computer device stores an operating system and computer readable instructions. The computer readable instructions are executed by the processor to implement an image correction method. The processor is used to provide computing and control capabilities to support the operation of the entire computer device. The internal memory in the computer device provides an environment for the operation of computer executable instructions in a non-volatile storage medium. The network interface is used for network communication with servers or other devices. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen. The input device may be a touch layer covered on the display screen, or may be a button, a trackball or a touchpad provided on a computer device casing, or may be An external keyboard, trackpad, or mouse. The computer device can be a cell phone, a tablet or a personal digital assistant or a wearable device or the like. It will be understood by those skilled in the art that the structure shown in FIG. 1 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied. The specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
图2为一个实施例中图像矫正方法的流程图。如图2所示,一种图像矫正方法,运行于计算机设备,该方法可以包括图2所示的操作。2 is a flow chart of an image correction method in one embodiment. As shown in FIG. 2, an image correction method is run on a computer device, and the method may include the operation shown in FIG. 2.
操作202,检测图像中包含脸部的形变轮廓。 Operation 202, detecting a deformation profile of the image including the face.
具体地,图像可为通过带摄像头的电子设备拍摄。该图像中包含有人脸。该图像可为存储在相册中或网络上的图像。形变轮廓是指脸部因屈光等而导致的脸部轮廓发生变形所形成的脸部轮廓。屈光是指因近视镜或远视镜导致眼镜区域有屈光。In particular, the image can be taken by an electronic device with a camera. The image contains a human face. The image can be an image stored in an album or on a network. The deformation profile refers to a facial contour formed by deformation of a facial contour caused by refraction or the like of the face. Refraction refers to the refraction of the area of the lens due to myopia or farsighted lenses.
对图像可采用机器学习模型进行检测得到图像中包含脸部的形变轮廓。为了通过机器学习模型识别,计算机设备需要预先收集正常的脸部轮廓样本和包含形变轮廓的脸部轮廓样本作为机器学习的训练样本,通过训练样本对机器学习模型进行训练得到脸部轮廓的机器学习模型。The image can be detected by a machine learning model to obtain a deformation profile containing the face in the image. In order to identify by machine learning model, the computer device needs to collect normal facial contour samples and facial contour samples containing deformation contours as training samples for machine learning, and train machine learning models through training samples to obtain machine learning of facial contours. model.
在一个实施例中,计算机设备也可通过人脸的关键特征点提取算法识别出图像中的人脸特征。人脸特征可包括眼、口、鼻、眉等几个特征。人脸关键特征点可包括2个眼球中心点、4个眼角点、2个鼻孔的中点和2个嘴角点。计算机设备可以采用susan算子提取局部区域的边缘和角点特征。Susan算子的原理为:以半径为像素的圆形区域为掩模,考察图像中的每个点在该区域 范围内的所有点的像素值与当前点的值的一致程度。在其他实施例中,计算机设备也可采用sobel、canny等边缘检测算子检测人脸的脸部轮廓及形变轮廓。In one embodiment, the computer device can also recognize facial features in the image through a key feature point extraction algorithm for the face. The facial features may include several features such as the eyes, mouth, nose, and eyebrows. The face key feature points may include 2 eyeball center points, 4 eye corner points, 2 nozzle midpoints, and 2 mouth corner points. The computer device can use the susan operator to extract the edge and corner features of the local region. The principle of the Susan operator is to use a circular area with a radius of pixels as a mask to examine how the pixel values of all points in the region of the image coincide with the values of the current point. In other embodiments, the computer device can also detect the facial contour and the deformation contour of the face by using an edge detection operator such as sobel or canny.
操作204,判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子。In operation 204, a deformation trend of the deformation profile is determined, and a corresponding deformation operator is selected according to the deformation trend.
具体地,计算机设备检测到包含脸部的形变轮廓后,可得到形变轮廓的形状。计算机设备将形变轮廓的形状与参考脸部轮廓的形状进行比较,可得到形变轮廓的形变趋势是缩小还是膨胀。参考脸部轮廓是指预先设置的作为标准的脸部轮廓。Specifically, after the computer device detects the deformation profile including the face, the shape of the deformation profile can be obtained. The computer device compares the shape of the deformed contour with the shape of the reference facial contour, and whether the deformation trend of the deformed contour is reduced or expanded. The reference face contour refers to a preset face contour as a standard.
在一个实施例中,操作204包括:当判断所述形变轮廓的形变趋势为缩小时,则选取第一形变算子;当判断所述形变轮廓的形变趋势为膨胀时,则选取第二形变算子。In one embodiment, the operation 204 includes: when determining that the deformation trend of the deformation profile is reduced, selecting a first deformation operator; and when determining that the deformation trend of the deformation profile is expanding, selecting a second deformation calculation child.
具体地,计算机设备可预先建立形变趋势与形变算子的对应关系,检测到形变趋势后,根据形变趋势从形变趋势与形变算子的对应关系获取对应的形变算子。形变算子是指对图像进行形变运算的参数。Specifically, the computer device may pre-establish a correspondence relationship between the deformation trend and the deformation operator, and after detecting the deformation trend, obtain a corresponding deformation operator from the corresponding relationship between the deformation trend and the deformation operator according to the deformation trend. A deformation operator is a parameter that performs a deformation operation on an image.
操作206,识别所述图像中所述脸部的脸部轮廓。 Operation 206, identifying a facial contour of the face in the image.
具体地,计算机设备可采用机器学习模型识别该图像中的脸部轮廓。该机器学习模型是预先通过训练样本训练得到的,或者通过人脸关键特征点提取得到。In particular, the computer device may employ a machine learning model to identify facial contours in the image. The machine learning model is obtained by training the training samples in advance, or by extracting key feature points of the face.
操作208,根据所述脸部轮廓对所述脸部的形变轮廓进行曲线拟合得到拟合曲线。 Operation 208, curve fitting the deformed contour of the face according to the facial contour to obtain a fitting curve.
具体地,计算机设备可以按照脸部轮廓中除了形变轮廓后剩余的其余轮廓拟合得到对应的参考脸部轮廓。计算机设备根据参考脸部轮廓与检测到图像中的脸部轮廓比较可得到形变轮廓进行曲线拟合后的脸部轮廓,即拟合曲线。Specifically, the computer device may fit the corresponding reference facial contour according to the remaining contours remaining in the facial contour except the deformed contour. The computer device obtains a curve contoured face contour, that is, a fitting curve, according to the reference face contour and the detected facial contour in the image.
曲线拟合可采用脸部轮廓中除形变轮廓外的散点,选择合适的曲线类型进行变量变换,使变换后的两个变量呈直线关系,按最小二乘法求线性方程 和方差,将直线化方程转换为关于原变量的函数表达式。Curve fitting can use scatter points other than the deformed contour in the contour of the face, select the appropriate curve type for variable transformation, make the two variables after the transformation have a linear relationship, and find the linear equation and variance according to the least squares method, and straighten the line. The equation is converted to a function expression about the original variable.
操作210,根据所述拟合曲线及所述形变轮廓对所述形变算子进行调整得到更新后的形变算子。In operation 210, the deformation operator is adjusted according to the fitting curve and the deformation profile to obtain an updated deformation operator.
具体地,计算机设备根据拟合曲线与形变轮廓两者之间的差别调整形变算子得到更新后的形变算子。该形变算子可为仿射变换矩阵。每个仿射变换对应一个矩形和一个向量的乘法。仿射变换可以通过一系列的原子变换的符合来实现,包括平移、缩放、翻转、旋转和错切。Specifically, the computer device adjusts the deformation operator according to the difference between the fitting curve and the deformation profile to obtain the updated deformation operator. The deformation operator can be an affine transformation matrix. Each affine transform corresponds to a multiplication of a rectangle and a vector. Affine transformations can be achieved through a series of atomic transformations, including translation, scaling, flipping, rotation, and miscutting.
例如,仿真变换采用3×3的矩阵来表示,其最后一行为(0,0,1)。该变换矩阵将原坐标(x 1,y 1)变换为新坐标(x 2,y 2),原坐标和新坐标均视最后一行为(1)的三维列相邻,原列向量左乘变换矩阵得到新的列向量,如公式(1)。 For example, the simulation transformation is represented by a 3 × 3 matrix, the last of which is (0, 0, 1). The transformation matrix transforms the original coordinates (x 1 , y 1 ) into new coordinates (x 2 , y 2 ). The original coordinates and the new coordinates are adjacent to the three-dimensional column of the last behavior (1), and the original column vector is multiplied by the transformation. The matrix gets a new column vector, as in equation (1).
Figure PCTCN2018094471-appb-000001
Figure PCTCN2018094471-appb-000001
为了将形变轮廓变为拟合曲线可将形变轮廓通过平移变换得到。平移变换的变换矩阵可为In order to change the deformation profile into a fitted curve, the deformation profile can be obtained by translational transformation. The transformation matrix of the translation transformation can be
Figure PCTCN2018094471-appb-000002
Figure PCTCN2018094471-appb-000002
其中,tx=x 2-x 1,ty=y 2-y 1。(x 2,y 2)为拟合曲线上某点的坐标,(x 1,y 1)为形变轮廓上对应点的坐标。 Where tx=x 2 -x 1 and ty=y 2 -y 1 . (x 2 , y 2 ) is the coordinate of a point on the fitted curve, and (x 1 , y 1 ) is the coordinate of the corresponding point on the deformed contour.
操作212,采用所述更新后的形变算子对所述形变轮廓进行形变处理。 Operation 212, deforming the deformation profile by using the updated deformation operator.
具体地,计算机设备通过更新后的形变算子对形变轮廓进行形变处理可得到矫正后的轮廓。Specifically, the computer device deforms the deformation profile by the updated deformation operator to obtain the corrected contour.
本申请实施例中的图像矫正方法,通过检测到图像中包含脸部的形变轮廓,根据形变轮廓的形变趋势选取对应的形变算子,检测到脸部轮廓,根据 脸部轮廓对形变轮廓进行拟合得到拟合曲线,根据拟合曲线及形变轮廓比较对形变算子进行调整得到更新后的形变算子,根据更新后的形变算子对形变轮廓进行矫正可得到矫正后的脸部轮廓,提高了面部的成像效果。The image correction method in the embodiment of the present application detects the deformation profile of the face in the image, selects a corresponding deformation operator according to the deformation trend of the deformation profile, detects the contour of the face, and formulates the deformation profile according to the contour of the face. The fitting curve is obtained, and the deformation operator is adjusted according to the fitting curve and the deformation contour to obtain the updated deformation operator, and the deformed contour can be corrected according to the updated deformation operator to obtain the corrected facial contour and improved. The imaging effect of the face.
需要说明的是,识别图像中的脸部轮廓可在操作202之前。It should be noted that identifying the facial contour in the image may precede the operation 202.
在一个实施例中,所述检测图像中包含脸部的形变轮廓包括:根据皮肤的颜色获取所述包含脸部的形变轮廓。In one embodiment, including the deformation profile of the face in the detection image comprises: acquiring the deformation profile including the face according to a color of the skin.
具体地,基于皮肤的颜色的人脸检测可包括预处理,基于肤色模型的肤色分割;连通域分析,人脸区域定位。预处理可采用高斯滤波和直方图均衡。肤色模型可采用YCbCr空间的色彩模型,其中,Y指亮度信息,Cb和Cr为色度信息。计算机设备根据肤色的均值和方差建立肤色的高斯模型,并通过肤色的高斯模型得到人脸概率图,使用二值化得到人脸肤色二值图像。计算机设备可以对输入的图像进行连通域分析获取二值图像最小外接矩形,即为人脸区域。具体可为:首先计算机设备对二值图像中符合预设连通规则的像素用相同的标号表示出来,得到二值图像的连通区域轮廓,求取连通区域的最小外接矩形。连通域标记的方法有像素标记法、线标记法、区域增长法等。以八领域像素标记法为例,1)判断此点八领域中的最左、左上、最上、上右点的情况,如果都没有点,则表示一个新的区域的开始;2)如果此点八领域中的最左有点,上右都有点,则标记此点为这两个中的最小的标记点,并修改大标记为小标记;3)如果此点八领域的左上有点,上右都有点,则标记此点为这两个中的最小标记点,并修改大标记为小标记;4)否则按照最左、左上、最上、上右的顺序,标记此点为四个中的一个。Specifically, face detection based on skin color may include pre-processing, skin color segmentation based on skin color model; connected domain analysis, face region localization. The preprocessing can use Gaussian filtering and histogram equalization. The skin color model can adopt a color model of the YCbCr space, where Y refers to luminance information, and Cb and Cr are chrominance information. The computer device establishes a Gaussian model of the skin color according to the mean and variance of the skin color, and obtains a face probability map through the Gaussian model of the skin color, and uses the binarization to obtain the face color binary image. The computer device can perform connected domain analysis on the input image to obtain a minimum circumscribed rectangle of the binary image, that is, a face region. Specifically, the computer device first displays the pixels in the binary image that meet the preset connection rule by the same label, obtains the connected area contour of the binary image, and obtains the minimum circumscribed rectangle of the connected area. Methods for connecting domain tags include pixel notation, line notation, and region growing. Taking the eight-field pixel marking method as an example, 1) judging the leftmost, upper left, uppermost, and upper right points in the eight fields of this point, if there is no point, it indicates the beginning of a new area; 2) if this point The leftmost point in the eight fields, there are points on the upper right, mark this point as the smallest mark in the two, and modify the big mark as a small mark; 3) If this point is the top left of the eight fields, the top right A little, mark this point as the smallest point in the two, and modify the big mark as a small mark; 4) Otherwise mark the point as one of the four in the order of leftmost, upper left, uppermost, upper right.
在一个实施例中,所述检测图像中包含脸部的形变轮廓包括:判断图像中是否存在眼镜,当图像中存在人脸时,则检测所述眼镜所在区域是否包含脸部,当眼镜所在区域包含脸部时,则获取所述眼镜所在区域的面部轮廓,将所述眼镜所在区域的面部轮廓作为所述包含脸部的形变轮廓。In one embodiment, the deforming contour of the detected image includes: determining whether there is glasses in the image, and when there is a human face in the image, detecting whether the area where the glasses is located includes a face, and when the glasses are located When the face is included, the facial contour of the area where the glasses are located is acquired, and the facial contour of the area where the glasses are located is used as the deformed contour including the face.
图3为另一个实施例中图像矫正方法的流程图。如图3所示,一种图像矫正方法,包括:3 is a flow chart of an image correction method in another embodiment. As shown in FIG. 3, an image correction method includes:
操作302,判断图像中是否存在眼镜,当图像中存在眼镜时,则执行操作304,当图像中不存在眼镜时,则结束。In operation 302, it is determined whether there is glasses in the image. When there are glasses in the image, operation 304 is performed, and when there is no glasses in the image, the process ends.
操作304,检测所述眼镜所在区域是否包含脸部轮廓,当所述眼镜所在区域包含脸部轮廓时,则执行操作306,当所述眼镜所在区域不包含脸部轮廓时,则结束。In operation 304, it is detected whether the area where the glasses are located includes a facial contour. When the area where the glasses is located includes a facial contour, operation 306 is performed, and when the area where the glasses is located does not include a facial contour, the processing ends.
操作306,判断眼镜所在区域的脸部轮廓的形变趋势是否为缩小,当形变趋势缩小时,则执行操作308,当形变趋势不为缩小时,则执行操作310。In operation 306, it is determined whether the deformation trend of the facial contour of the region where the glasses are located is reduced. When the deformation trend is reduced, operation 308 is performed, and when the deformation trend is not reduced, operation 310 is performed.
操作308,选取第一形变算子,执行操作312。 Operation 308, selecting the first deformation operator, and performing operation 312.
第一形变算子为近视镜形变算子。The first deformation operator is a myopia deformation operator.
操作310,选取第二形变算子,执行操作312。 Operation 310, selecting the second deformation operator, performs operation 312.
第二形变算子为远视镜形变算子。The second deformation operator is a far vision mirror deformation operator.
操作312,识别所述图像中所述脸部的脸部轮廓。 Operation 312, identifying a facial contour of the face in the image.
操作314,根据所述脸部轮廓对所述脸部的形变轮廓进行曲线拟合得到拟合曲线。 Operation 314, curve fitting the deformed contour of the face according to the facial contour to obtain a fitting curve.
操作316,根据所述拟合曲线及所述形变轮廓对所述形变算子进行调整得到更新后的形变算子。 Operation 316, adjusting the deformation operator according to the fitting curve and the deformation profile to obtain an updated deformation operator.
操作318,采用所述更新后的形变算子对所述形变轮廓进行形变处理。 Operation 318, deforming the deformation profile by using the updated deformation operator.
本申请实施例中的图像矫正方法,通过检测到图像中眼部所在区域包含脸部轮廓,根据眼镜所在区域中脸部轮廓的形变趋势选取对应的形变算子,检测到脸部轮廓,根据脸部轮廓对形变轮廓进行拟合得到拟合曲线,根据拟合曲线及形变轮廓比较对形变算子进行调整得到更新后的形变算子,根据更新后的形变算子对形变轮廓进行矫正可得到矫正后的脸部轮廓,提高了面部的成像效果,让戴眼镜的用户拍摄时得到较好的人像照片。当眼部所在区域不包含脸部区域时,则结束,可减少数据处理。In the image correction method in the embodiment of the present application, by detecting that the region where the eye is located in the image includes a facial contour, the corresponding deformation operator is selected according to the deformation trend of the facial contour in the region where the glasses are located, and the facial contour is detected according to the face. The contour of the contour is fitted to the deformation profile to obtain a fitting curve. The deformation operator is adjusted according to the fitting curve and the deformation contour to obtain the updated deformation operator. The deformation deformation can be corrected according to the updated deformation operator. The contour of the back face improves the imaging effect of the face, so that the user wearing the glasses gets a better portrait photo when shooting. When the area where the eye is located does not contain the face area, it ends, which reduces data processing.
图4为一个实施例中戴眼镜用户拍照发生形变的示意图。如图4所示,因眼镜的屈光导致眼镜所在区域的脸部轮廓向内凹,如脸部轮廓42和脸部轮廓44两者存在断层,则脸部轮廓44表示形变轮廓。FIG. 4 is a schematic diagram showing deformation of a photo taken by a user wearing glasses in an embodiment. FIG. As shown in FIG. 4, the facial contour of the area in which the glasses are located is inwardly concave due to the refraction of the glasses, and if there is a fault in both the facial contour 42 and the facial contour 44, the facial contour 44 represents the deformed contour.
图5为一个实施例中对图4中的脸部轮廓中形变区域进行拟合得到拟合曲线的示意图。如图5所示,对脸部轮廓42和脸部轮廓44之间的断层区域进行拟合得到拟合曲线46。根据拟合曲线46和脸部轮廓44可对选取的形变算子进行调整得到更新后的形变算子。根据更新后的形变算子对脸部轮廓44进行形变处理得到矫正的脸部轮廓。FIG. 5 is a schematic diagram showing a fitting curve obtained by fitting a deformation region in the contour of the face in FIG. 4 in one embodiment. FIG. As shown in FIG. 5, a fitted curve 46 is obtained by fitting a fault region between the facial contour 42 and the facial contour 44. The selected deformation operator can be adjusted according to the fitting curve 46 and the face contour 44 to obtain an updated deformation operator. The facial contour 44 is deformed according to the updated deformation operator to obtain a corrected facial contour.
此外,上述图像矫正方法可应用于照片编辑器中。在照片编辑器中采用该图像矫正方法对照片进行矫正。Furthermore, the above image correction method can be applied to a photo editor. The image correction method is used to correct the photo in the photo editor.
本申请实施例的方法流程图中的各个操作按照箭头的指示依次显示,但是这些操作并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明,这些操作的执行并没有严格的顺序限制,其可以以其他的顺序执行。而且,本申请实施例的方法流程图中的至少一部分操作可以包括多个子操作或者多个阶段,这些子操作或者阶段并不必然是在同一时刻执行完成,而是可以在不同的时刻执行,其执行顺序也不必然是依次进行,而是可以与其他操作或者其他操作的子操作或者阶段的至少一部分轮流或者交替地执行。The operations in the flowchart of the method of the embodiment of the present application are sequentially displayed in accordance with the indication of the arrows, but the operations are not necessarily performed in the order indicated by the arrows. Except as explicitly stated herein, the execution of these operations is not strictly limited, and may be performed in other sequences. Moreover, at least a part of the operations in the method flowchart of the embodiment of the present application may include multiple sub-operations or multiple stages, which are not necessarily performed at the same time, but may be executed at different times. The order of execution is not necessarily performed sequentially, but may be performed alternately or alternately with at least a portion of the sub-operations or phases of other operations or other operations.
图6为一个实施例中图像矫正装置的内部框图。如图6所示,一种图像矫正装置600,包括检测模块602、选取模块604、识别模块606、拟合模块608、调整模块610和矫正模块612。其中:Figure 6 is an internal block diagram of an image correcting device in one embodiment. As shown in FIG. 6 , an image correction device 600 includes a detection module 602 , a selection module 604 , an identification module 606 , a fitting module 608 , an adjustment module 610 , and a correction module 612 . among them:
检测模块602检测图像中包含脸部的形变轮廓。The detection module 602 detects a deformation profile that includes a face in the image.
选取模块604用于判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子。The selecting module 604 is configured to determine a deformation trend of the deformation profile, and select a corresponding deformation operator according to the deformation trend.
识别模块606用于识别所述图像中所述脸部的脸部轮廓。The identification module 606 is for identifying a facial contour of the face in the image.
拟合模块608用于根据所述脸部轮廓对所述脸部的形变轮廓进行曲线拟合得到拟合曲线。The fitting module 608 is configured to perform curve fitting on the deformation profile of the face according to the facial contour to obtain a fitting curve.
调整模块610用于根据所述拟合曲线及所述形变轮廓对所述形变算子进行调整得到更新后的形变算子。The adjustment module 610 is configured to adjust the deformation operator according to the fitting curve and the deformation profile to obtain an updated deformation operator.
矫正模块612用于采用所述更新后的形变算子对所述形变轮廓进行形变 处理。The correction module 612 is configured to deform the deformation profile by using the updated deformation operator.
本申请实施例中的图像矫正装置,通过检测到图像中包含脸部的形变轮廓,根据形变轮廓的形变趋势选取对应的形变算子,检测到脸部轮廓,根据脸部轮廓对形变轮廓进行拟合得到拟合曲线,根据拟合曲线及形变轮廓比较对形变算子进行调整得到更新后的形变算子,根据更新后的形变算子对形变轮廓进行矫正可得到矫正后的脸部轮廓,提高了面部的成像效果。The image correcting device in the embodiment of the present application detects the deformation profile of the face in the image, selects a corresponding deformation operator according to the deformation trend of the deformation profile, detects the contour of the face, and formulates the deformation profile according to the contour of the face. The fitting curve is obtained, and the deformation operator is adjusted according to the fitting curve and the deformation contour to obtain the updated deformation operator, and the deformed contour can be corrected according to the updated deformation operator to obtain the corrected facial contour and improved. The imaging effect of the face.
在一个实施例中,检测模块602还用于根据皮肤的颜色获取所述包含脸部的形变轮廓。In one embodiment, the detection module 602 is further configured to acquire the deformation profile including the face according to the color of the skin.
在一个实施例中,检测模块602还用于判断图像中是否存在眼镜,当图像中存在眼镜时,则检测所述眼镜所在区域是否包含脸部,当眼镜所在区域包含脸部时,则获取所述眼镜所在区域的面部轮廓,将所述眼镜所在区域的面部轮廓作为所述包含脸部的形变轮廓。In an embodiment, the detecting module 602 is further configured to determine whether there is glasses in the image, and when there is glasses in the image, detecting whether the area where the glasses is located includes a face, and when the area where the glasses is located includes a face, acquiring the The facial contour of the region where the glasses are located, and the facial contour of the region where the glasses are located is used as the deformed contour including the face.
在一个实施例中,选取模块604还用于当判断所述形变轮廓的形变趋势为缩小时,则选取第一形变算子;当判断所述形变轮廓的形变趋势为膨胀时,则选取第二形变算子。In an embodiment, the selecting module 604 is further configured to: when determining that the deformation trend of the deformation contour is reduced, selecting a first deformation operator; and when determining that the deformation trend of the deformation contour is expanding, selecting the second Deformation operator.
在一个实施例中,检测模块602还用于采用机器学习模型识别图像中包含脸部的形变轮廓。In one embodiment, the detection module 602 is further configured to identify a deformation profile of the image that includes the face using a machine learning model.
上述图像矫正装置中各个模块的划分仅用于举例说明,在其他实施例中,可将图像矫正装置按照需要划分为不同的模块,以完成上述图像矫正装置的全部或部分功能。The division of each module in the image correcting device described above is for illustrative purposes only. In other embodiments, the image correcting device may be divided into different modules as needed to perform all or part of the functions of the image correcting device.
上述图像矫正装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于服务器中的处理器中,也可以以软件形式存储于服务器中的存储器中,以便于处理器调用执行以上各个模块对应的操作。如在本申请中所使用的,术语“组件”、“模块”和“系统”等旨在表示计算机相关的实体,它可以是硬件、硬件和软件的组合、软件、或者执行中的软件。例如,组件可以是但不限于是,在处理器上运行的进程、处理器、对象、可执行码、执行的线程、程序和/或计算机。作为说明, 运行在服务器上的应用程序和服务器都可以是组件。一个或多个组件可以驻留在进程和/或执行的线程中,并且组件可以位于一个计算机内和/或分布在两个或更多的计算机之间。The various modules in the image correcting device described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in the hardware in the processor or in the memory in the server, or may be stored in the memory in the server, so that the processor calls the corresponding operations of the above modules. As used in this application, the terms "component", "module" and "system" and the like are intended to mean a computer-related entity, which may be hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. As an illustration, both the application running on the server and the server can be components. One or more components can reside within a process and/or executed thread, and the components can be located within one computer and/or distributed between two or more computers.
本申请实施例还提供了一种非易失性计算机可读存储介质。一个或多个包含计算机可执行指令的非易失性计算机可读存储介质,当所述计算机可执行指令被一个或多个处理器执行时,使得所述处理器执行如上所述图像矫正方法。The embodiment of the present application also provides a non-transitory computer readable storage medium. One or more non-transitory computer readable storage media containing computer executable instructions that, when executed by one or more processors, cause the processor to perform an image correction method as described above.
本申请实施例还提供一种计算机设备。上述计算机设备中包括图像处理电路,图像处理电路可以利用硬件和/或软件组件实现,可包括定义ISP(Image Signal Processing,图像信号处理)管线的各种处理单元。图7为一个实施例中图像处理电路的示意图。如图7所示,为便于说明,仅示出与本申请实施例相关的图像处理技术的各个方面。The embodiment of the present application further provides a computer device. The above computer device includes an image processing circuit, and the image processing circuit may be implemented by hardware and/or software components, and may include various processing units defining an ISP (Image Signal Processing) pipeline. Figure 7 is a schematic illustration of an image processing circuit in one embodiment. As shown in FIG. 7, for convenience of explanation, only various aspects of the image processing technique related to the embodiment of the present application are shown.
如图7所示,图像处理电路包括ISP处理器740和控制逻辑器750。成像设备710捕捉的图像数据首先由ISP处理器740处理,ISP处理器740对图像数据进行分析以捕捉可用于确定和/或成像设备710的一个或多个控制参数的图像统计信息。成像设备710可包括具有一个或多个透镜712和图像传感器714的照相机。图像传感器714可包括色彩滤镜阵列(如Bayer滤镜),图像传感器714可获取用图像传感器714的每个成像像素捕捉的光强度和波长信息,并提供可由ISP处理器740处理的一组原始图像数据。传感器720可基于传感器720接口类型把原始图像数据提供给ISP处理器740。传感器720接口可以利用SMIA(Standard Mobile Imaging Architecture,标准移动成像架构)接口、其它串行或并行照相机接口或上述接口的组合。As shown in FIG. 7, the image processing circuit includes an ISP processor 740 and a control logic 750. The image data captured by imaging device 710 is first processed by ISP processor 740, which analyzes the image data to capture image statistical information that may be used to determine and/or control one or more control parameters of imaging device 710. Imaging device 710 can include a camera having one or more lenses 712 and image sensors 714. Image sensor 714 can include a color filter array (such as a Bayer filter) that can capture light intensity and wavelength information captured with each imaging pixel of image sensor 714 and provide a set of primitives that can be processed by ISP processor 740 Image data. Sensor 720 can provide raw image data to ISP processor 740 based on sensor 720 interface type. The sensor 720 interface may utilize a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above.
ISP处理器740按多种格式逐个像素地处理原始图像数据。例如,每个图像像素可具有8、10、12或14比特的位深度,ISP处理器740可对原始图像数据进行一个或多个图像处理操作、收集关于图像数据的统计信息。其中,图像处理操作可按相同或不同的位深度精度进行。The ISP processor 740 processes the raw image data pixel by pixel in a variety of formats. For example, each image pixel can have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 740 can perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Among them, image processing operations can be performed with the same or different bit depth precision.
ISP处理器740还可从图像存储器730接收像素数据。例如,从传感器 720接口将原始像素数据发送给图像存储器730,图像存储器730中的原始像素数据再提供给ISP处理器740以供处理。图像存储器730可为存储器装置的一部分、存储设备、或电子设备内的独立的专用存储器,并可包括DMA(Direct Memory Access,直接直接存储器存取)特征。 ISP processor 740 can also receive pixel data from image memory 730. For example, raw pixel data is sent from the sensor 720 interface to image memory 730, which is then provided to ISP processor 740 for processing. Image memory 730 can be part of a memory device, a storage device, or a separate dedicated memory within an electronic device, and can include DMA (Direct Memory Access) features.
当接收到来自传感器720接口或来自图像存储器730的原始图像数据时,ISP处理器740可进行一个或多个图像处理操作,如时域滤波。处理后的图像数据可发送给图像存储器730,以便在被显示之前进行另外的处理。还可直接从ISP处理器740接收“前端”处理数据,或从图像存储器730接收“前端”处理数据,并对“前端”处理数据进行原始域中以及RGB和YCbCr颜色空间中的图像数据处理。处理后的图像数据可输出给显示器770,以供用户观看和/或由图形引擎或GPU(Graphics Processing Unit,图形处理器)进一步处理。此外,ISP处理器740的输出还可发送给图像存储器730,且显示器770可从图像存储器730读取图像数据。在一个实施例中,图像存储器730可被配置为实现一个或多个帧缓冲器。此外,ISP处理器740的输出可发送给编码器/解码器760,以便编码/解码图像数据。编码的图像数据可被保存,并在显示于显示器770设备上之前解压缩。When receiving raw image data from sensor 720 interface or from image memory 730, ISP processor 740 can perform one or more image processing operations, such as time domain filtering. The processed image data can be sent to image memory 730 for additional processing before being displayed. The "front end" processing data may also be received directly from the ISP processor 740, or the "front end" processing data may be received from the image memory 730, and the "front end" processing data may be processed in the original domain and in the RGB and YCbCr color spaces. The processed image data can be output to display 770 for viewing by a user and/or further processed by a graphics engine or GPU (Graphics Processing Unit). Additionally, the output of ISP processor 740 can also be sent to image memory 730, and display 770 can read image data from image memory 730. In one embodiment, image memory 730 can be configured to implement one or more frame buffers. Additionally, the output of ISP processor 740 can be sent to encoder/decoder 760 to encode/decode image data. The encoded image data can be saved and decompressed before being displayed on the display 770 device.
ISP处理器740处理图像数据的操作包括:对图像数据进行VFE(Video Front End,视频前端)处理和CPP(Camera Post Processing,摄像头后处理)处理。对图像数据的VFE处理可包括修正图像数据的对比度或亮度、修改以数字方式记录的光照状态数据、对图像数据进行补偿处理(如白平衡,自动增益控制,γ校正等)、对图像数据进行滤波处理等。对图像数据的CPP处理可包括对图像进行缩放、向每个路径提供预览帧和记录帧。其中,CPP可使用不同的编解码器来处理预览帧和记录帧。ISP处理器740处理后的图像数据可发送给美颜模块760,以便在被显示之前对图像进行美颜处理。美颜模块760对图像数据美颜处理可包括:美白、祛斑、磨皮、瘦脸、祛痘、增大眼睛等。其中,美颜模块760可为计算机设备中CPU(Central Processing Unit,中央处理器)或GPU(Graphics Processing Unit,图形处理器)等。美颜模块 760处理后的数据可发送给编码器/解码器770,以便编码/解码图像数据。编码的图像数据可被保存,并在显示与显示器780设备上之前解压缩。The ISP processor 740 processes the image data by performing VFE (Video Front End) processing and CPP (Camera Post Processing) processing on the image data. The VFE processing of the image data may include correcting the contrast or brightness of the image data, modifying the digitally recorded illumination state data, performing compensation processing on the image data (such as white balance, automatic gain control, gamma correction, etc.), and performing image data. Filter processing, etc. CPP processing of image data may include scaling the image, providing a preview frame and a recording frame to each path. Among them, CPP can use different codecs to process preview frames and record frames. The image data processed by the ISP processor 740 can be sent to the beauty module 760 for aesthetic processing of the image prior to being displayed. The beauty treatment of the image data by the beauty module 760 may include: whitening, freckle, dermabrasion, face-lifting, acne, eye enlargement, and the like. The beauty module 760 can be a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) in a computer device. The processed data of the beauty module 760 can be sent to the encoder/decoder 770 to encode/decode the image data. The encoded image data can be saved and decompressed before being displayed on the display 780 device.
ISP处理器740确定的统计数据可发送给控制逻辑器750单元。例如,统计数据可包括自动曝光、自动白平衡、自动聚焦、闪烁检测、黑电平补偿、透镜712阴影校正等图像传感器714统计信息。控制逻辑器750可包括执行一个或多个例程(如固件)的处理器和/或微控制器,一个或多个例程可根据接收的统计数据,确定成像设备710的控制参数以及的控制参数。例如,控制参数可包括传感器720控制参数(例如增益、曝光控制的积分时间)、照相机闪光控制参数、透镜712控制参数(例如聚焦或变焦用焦距)、或这些参数的组合。ISP控制参数可包括用于自动白平衡和颜色调整(例如,在RGB处理期间)的增益水平和色彩校正矩阵,以及透镜712阴影校正参数。The statistics determined by the ISP processor 740 can be sent to the control logic 750 unit. For example, the statistics may include image sensor 714 statistics such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, lens 712 shading correction, and the like. Control logic 750 can include a processor and/or a microcontroller that executes one or more routines, such as firmware, and one or more routines can determine control parameters and control of imaging device 710 based on received statistical data. parameter. For example, the control parameters may include sensor 720 control parameters (eg, gain, integration time for exposure control), camera flash control parameters, lens 712 control parameters (eg, focus or zoom focal length), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), and lens 712 shading correction parameters.
通过图7中图像处理技术中处理器实现上述图像矫正方法。The image correction method described above is implemented by a processor in the image processing technique of FIG.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机程序来指令相关的硬件来完成,所述的程序可存储于一非易失性计算机可读取存储介质中,该程序在执行时,可包括如上述各方法的实施例的流程。其中,所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory,ROM)等。One of ordinary skill in the art can understand that all or part of the process of implementing the above embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a non-volatile computer readable storage medium. Wherein, the program, when executed, may include the flow of an embodiment of the methods as described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or the like.
以上所述实施例仅表达了本申请的几种实施方式,其描述较为具体和详细,但并不能因此而理解为对本申请专利范围的限制。应当指出的是,对于本领域的普通技术人员来说,在不脱离本申请构思的前提下,还可以做出若干变形和改进,这些都属于本申请的保护范围。因此,本申请专利的保护范围应以所附权利要求为准。The above-mentioned embodiments are merely illustrative of several embodiments of the present application, and the description thereof is more specific and detailed, but is not to be construed as limiting the scope of the claims. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the present application. Therefore, the scope of the invention should be determined by the appended claims.

Claims (18)

  1. 一种图像矫正方法,包括:An image correction method comprising:
    检测图像中包含脸部的形变轮廓;Detecting a deformation profile of the face in the image;
    判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子;Determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend;
    识别所述图像中所述脸部的脸部轮廓;Identifying a facial contour of the face in the image;
    根据所述脸部轮廓对所述脸部的形变轮廓进行曲线拟合得到拟合曲线;Performing a curve fitting on the deformation profile of the face according to the facial contour to obtain a fitting curve;
    根据所述拟合曲线及所述形变轮廓对所述形变算子进行调整得到更新后的形变算子;及Adjusting the deformation operator according to the fitting curve and the deformation profile to obtain an updated deformation operator; and
    采用所述更新后的形变算子对所述形变轮廓进行形变处理。The deformation profile is subjected to deformation processing using the updated deformation operator.
  2. 根据权利要求1所述的方法,其特征在于,所述检测图像中包含脸部的形变轮廓包括:The method according to claim 1, wherein the deformation profile including the face in the detected image comprises:
    根据皮肤的颜色获取所述包含脸部的形变轮廓。The deformation profile including the face is obtained according to the color of the skin.
  3. 根据权利要求1所述的方法,其特征在于,所述检测图像中包含脸部的形变轮廓包括:The method according to claim 1, wherein the deformation profile including the face in the detected image comprises:
    判断图像中是否存在眼镜,当所述图像中存在眼镜时,则检测所述眼镜所在区域是否包含脸部;Determining whether there is glasses in the image, and when there are glasses in the image, detecting whether the area where the glasses are located includes a face;
    当所述眼镜所在区域包含脸部时,则获取所述眼镜所在区域的面部轮廓;及When the area where the glasses are located includes a face, the facial contour of the area where the glasses are located is acquired; and
    将所述眼镜所在区域的面部轮廓作为所述包含脸部的形变轮廓。The facial contour of the region in which the glasses are located is taken as the deformed contour including the face.
  4. 根据权利要求1所述的方法,其特征在于,所述判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子包括:The method according to claim 1, wherein the determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend comprises:
    当判断所述形变轮廓的形变趋势为缩小时,则选取第一形变算子。When it is judged that the deformation tendency of the deformation profile is reduced, the first deformation operator is selected.
  5. 根据权利要求1所述的方法,其特征在于,所述判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子包括:The method according to claim 1, wherein the determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend comprises:
    当判断所述形变轮廓的形变趋势为膨胀时,则选取第二形变算子。When it is judged that the deformation tendency of the deformation profile is expansion, the second deformation operator is selected.
  6. 根据权利要求1所述的方法,其特征在于,所述检测图像中包含脸部的形变轮廓包括:The method according to claim 1, wherein the deformation profile including the face in the detected image comprises:
    采用机器学习模型识别图像中包含脸部的形变轮廓。A machine learning model is used to identify the deformation profile of the image that contains the face.
  7. 一个或多个包含计算机可执行指令的非易失性计算机可读存储介质,当所述计算机可执行指令被一个或多个处理器执行时,使得所述处理器执行以下操作:One or more non-transitory computer readable storage media containing computer executable instructions that, when executed by one or more processors, cause the processor to:
    检测图像中包含脸部的形变轮廓;Detecting a deformation profile of the face in the image;
    判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子;Determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend;
    识别所述图像中所述脸部的脸部轮廓;Identifying a facial contour of the face in the image;
    根据所述脸部轮廓对所述脸部的形变轮廓进行曲线拟合得到拟合曲线;Performing a curve fitting on the deformation profile of the face according to the facial contour to obtain a fitting curve;
    根据所述拟合曲线及所述形变轮廓对所述形变算子进行调整得到更新后的形变算子;及Adjusting the deformation operator according to the fitting curve and the deformation profile to obtain an updated deformation operator; and
    采用所述更新后的形变算子对所述形变轮廓进行形变处理。The deformation profile is subjected to deformation processing using the updated deformation operator.
  8. 根据权利要求7所述的非易失性计算机可读存储介质,其特征在于,所述检测图像中包含脸部的形变轮廓包括:The non-transitory computer readable storage medium according to claim 7, wherein the deformation profile including the face in the detected image comprises:
    根据皮肤的颜色获取所述包含脸部的形变轮廓。The deformation profile including the face is obtained according to the color of the skin.
  9. 根据权利要求7所述的非易失性计算机可读存储介质,其特征在于,所述检测图像中包含脸部的形变轮廓包括:The non-transitory computer readable storage medium according to claim 7, wherein the deformation profile including the face in the detected image comprises:
    判断图像中是否存在眼镜,当所述图像中存在眼镜时,则检测所述眼镜所在区域是否包含脸部;Determining whether there is glasses in the image, and when there are glasses in the image, detecting whether the area where the glasses are located includes a face;
    当所述眼镜所在区域包含脸部时,则获取所述眼镜所在区域的面部轮廓;及When the area where the glasses are located includes a face, the facial contour of the area where the glasses are located is acquired; and
    将所述眼镜所在区域的面部轮廓作为所述包含脸部的形变轮廓。The facial contour of the region in which the glasses are located is taken as the deformed contour including the face.
  10. 根据权利要求7所述的非易失性计算机可读存储介质,其特征在于,所述判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子包括:The non-transitory computer readable storage medium according to claim 7, wherein the determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend comprises:
    当判断所述形变轮廓的形变趋势为缩小时,则选取第一形变算子。When it is judged that the deformation tendency of the deformation profile is reduced, the first deformation operator is selected.
  11. 根据权利要求7所述的非易失性计算机可读存储介质,其特征在于,所述判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子 包括:The non-transitory computer readable storage medium according to claim 7, wherein the determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend comprises:
    当判断所述形变轮廓的形变趋势为膨胀时,则选取第二形变算子。When it is judged that the deformation tendency of the deformation profile is expansion, the second deformation operator is selected.
  12. 根据权利要求7所述的非易失性计算机可读存储介质,其特征在于,所述检测图像中包含脸部的形变轮廓包括:The non-transitory computer readable storage medium according to claim 7, wherein the deformation profile including the face in the detected image comprises:
    采用机器学习模型识别图像中包含脸部的形变轮廓。A machine learning model is used to identify the deformation profile of the image that contains the face.
  13. 一种计算机设备,包括存储器及处理器,所述存储器中储存有计算机可读指令,所述指令被所述处理器执行时,使得所述处理器执行以下操作:A computer device comprising a memory and a processor, the memory storing computer readable instructions, wherein when executed by the processor, the processor causes the processor to:
    检测图像中包含脸部的形变轮廓;Detecting a deformation profile of the face in the image;
    判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子;Determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend;
    识别所述图像中所述脸部的脸部轮廓;Identifying a facial contour of the face in the image;
    根据所述脸部轮廓对所述脸部的形变轮廓进行曲线拟合得到拟合曲线;Performing a curve fitting on the deformation profile of the face according to the facial contour to obtain a fitting curve;
    根据所述拟合曲线及所述形变轮廓对所述形变算子进行调整得到更新后的形变算子;及Adjusting the deformation operator according to the fitting curve and the deformation profile to obtain an updated deformation operator; and
    采用所述更新后的形变算子对所述形变轮廓进行形变处理。The deformation profile is subjected to deformation processing using the updated deformation operator.
  14. 根据权利要求13所述的计算机设备,其特征在于,所述检测图像中包含脸部的形变轮廓包括:The computer device according to claim 13, wherein the deformation profile including the face in the detected image comprises:
    根据皮肤的颜色获取所述包含脸部的形变轮廓。The deformation profile including the face is obtained according to the color of the skin.
  15. 根据权利要求13所述的计算机设备,其特征在于,所述检测图像中包含脸部的形变轮廓包括:The computer device according to claim 13, wherein the deformation profile including the face in the detected image comprises:
    判断图像中是否存在眼镜,当所述图像中存在眼镜时,则检测所述眼镜所在区域是否包含脸部;Determining whether there is glasses in the image, and when there are glasses in the image, detecting whether the area where the glasses are located includes a face;
    当所述眼镜所在区域包含脸部时,则获取所述眼镜所在区域的面部轮廓;及When the area where the glasses are located includes a face, the facial contour of the area where the glasses are located is acquired; and
    将所述眼镜所在区域的面部轮廓作为所述包含脸部的形变轮廓。The facial contour of the region in which the glasses are located is taken as the deformed contour including the face.
  16. 根据权利要求13所述的计算机设备,其特征在于,所述判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子包括:The computer device according to claim 13, wherein the determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend comprises:
    当判断所述形变轮廓的形变趋势为缩小时,则选取第一形变算子。When it is judged that the deformation tendency of the deformation profile is reduced, the first deformation operator is selected.
  17. 根据权利要求13所述的计算机设备,其特征在于,所述判断所述形变轮廓的形变趋势,根据所述形变趋势选取对应的形变算子包括:The computer device according to claim 13, wherein the determining a deformation trend of the deformation profile, and selecting a corresponding deformation operator according to the deformation trend comprises:
    当判断所述形变轮廓的形变趋势为膨胀时,则选取第二形变算子。When it is judged that the deformation tendency of the deformation profile is expansion, the second deformation operator is selected.
  18. 根据权利要求13所述的计算机设备,其特征在于,所述检测图像中包含脸部的形变轮廓包括:The computer device according to claim 13, wherein the deformation profile including the face in the detected image comprises:
    采用机器学习模型识别图像中包含脸部的形变轮廓。A machine learning model is used to identify the deformation profile of the image that contains the face.
PCT/CN2018/094471 2017-07-18 2018-07-04 Image correction method, computer readable storage medium and computer device WO2019015477A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710587813.9A CN107368806B (en) 2017-07-18 2017-07-18 Image rectification method, image rectification device, computer-readable storage medium and computer equipment
CN201710587813.9 2017-07-18

Publications (1)

Publication Number Publication Date
WO2019015477A1 true WO2019015477A1 (en) 2019-01-24

Family

ID=60306903

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/094471 WO2019015477A1 (en) 2017-07-18 2018-07-04 Image correction method, computer readable storage medium and computer device

Country Status (2)

Country Link
CN (1) CN107368806B (en)
WO (1) WO2019015477A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112529927A (en) * 2020-12-11 2021-03-19 西安电子科技大学 Self-adaptive contour extraction system and method based on FPGA morphological operator
CN112767359A (en) * 2021-01-21 2021-05-07 中南大学 Steel plate corner detection method and system under complex background
CN115063607A (en) * 2022-08-17 2022-09-16 腾讯科技(深圳)有限公司 Method, device, equipment, storage medium and program product for determining cutting contour
CN117935068A (en) * 2024-03-25 2024-04-26 中国平安财产保险股份有限公司四川分公司 Crop disease analysis method and analysis system

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107368806B (en) * 2017-07-18 2020-01-10 Oppo广东移动通信有限公司 Image rectification method, image rectification device, computer-readable storage medium and computer equipment
CN109447911B (en) * 2018-10-18 2020-06-09 百度在线网络技术(北京)有限公司 Image restoration method, device, storage medium and terminal equipment
CN111444856A (en) * 2020-03-27 2020-07-24 广东博智林机器人有限公司 Image analysis method, model training method, device, equipment and storage medium
CN113486714B (en) * 2021-06-03 2022-09-02 荣耀终端有限公司 Image processing method and electronic equipment
CN113435445A (en) * 2021-07-05 2021-09-24 深圳市鹰硕技术有限公司 Image over-optimization automatic correction method and device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103824253A (en) * 2014-02-19 2014-05-28 中山大学 Figure five sense organ deformation method based on image local precise deformation
JP2014146215A (en) * 2013-01-30 2014-08-14 Nec Corp Image correction device, image correction method and image correction program
CN105678251A (en) * 2015-12-31 2016-06-15 Tcl海外电子(惠州)有限公司 Face image processing method and device
CN107368806A (en) * 2017-07-18 2017-11-21 广东欧珀移动通信有限公司 Image correction method, device, computer-readable recording medium and computer equipment

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100464346C (en) * 2006-04-29 2009-02-25 北大方正集团有限公司 Digital correction method for geometric distortion of form image
CN102034079B (en) * 2009-09-24 2012-11-28 汉王科技股份有限公司 Method and system for identifying faces shaded by eyeglasses
CN102542534B (en) * 2010-12-31 2015-01-14 北京海思威科技有限公司 Image distortion correcting method and device based on image contour
CN104811602A (en) * 2014-01-24 2015-07-29 维科技术有限公司 Self-shooting method and self-shooting device for mobile terminals
CN104408426B (en) * 2014-11-27 2018-07-24 小米科技有限责任公司 Facial image glasses minimizing technology and device
CN104537630A (en) * 2015-01-22 2015-04-22 厦门美图之家科技有限公司 Method and device for image beautifying based on age estimation
US9514524B2 (en) * 2015-02-27 2016-12-06 Sony Corporation Optical distortion compensation

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014146215A (en) * 2013-01-30 2014-08-14 Nec Corp Image correction device, image correction method and image correction program
CN103824253A (en) * 2014-02-19 2014-05-28 中山大学 Figure five sense organ deformation method based on image local precise deformation
CN105678251A (en) * 2015-12-31 2016-06-15 Tcl海外电子(惠州)有限公司 Face image processing method and device
CN107368806A (en) * 2017-07-18 2017-11-21 广东欧珀移动通信有限公司 Image correction method, device, computer-readable recording medium and computer equipment

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112529927A (en) * 2020-12-11 2021-03-19 西安电子科技大学 Self-adaptive contour extraction system and method based on FPGA morphological operator
CN112767359A (en) * 2021-01-21 2021-05-07 中南大学 Steel plate corner detection method and system under complex background
CN112767359B (en) * 2021-01-21 2023-10-24 中南大学 Method and system for detecting corner points of steel plate under complex background
CN115063607A (en) * 2022-08-17 2022-09-16 腾讯科技(深圳)有限公司 Method, device, equipment, storage medium and program product for determining cutting contour
CN117935068A (en) * 2024-03-25 2024-04-26 中国平安财产保险股份有限公司四川分公司 Crop disease analysis method and analysis system
CN117935068B (en) * 2024-03-25 2024-05-24 中国平安财产保险股份有限公司四川分公司 Crop disease analysis method and analysis system

Also Published As

Publication number Publication date
CN107368806A (en) 2017-11-21
CN107368806B (en) 2020-01-10

Similar Documents

Publication Publication Date Title
WO2019015477A1 (en) Image correction method, computer readable storage medium and computer device
US10929646B2 (en) Method and apparatus for image processing, and computer-readable storage medium
EP3477931B1 (en) Image processing method and device, readable storage medium and electronic device
US11250571B2 (en) Robust use of semantic segmentation in shallow depth of field rendering
JP7027537B2 (en) Image processing methods and equipment, electronic devices, and computer-readable storage media
WO2021017811A1 (en) Image processing method and apparatus, electronic device, and computer readable storage medium
CN108537155B (en) Image processing method, image processing device, electronic equipment and computer readable storage medium
WO2017101626A1 (en) Method and apparatus for implementing image processing
EP3480784B1 (en) Image processing method, and device
US11854243B2 (en) Gaze correction of multi-view images
CN107766831B (en) Image processing method, image processing device, mobile terminal and computer-readable storage medium
CN107509031B (en) Image processing method, image processing device, mobile terminal and computer readable storage medium
WO2021022983A1 (en) Image processing method and apparatus, electronic device and computer-readable storage medium
US11538175B2 (en) Method and apparatus for detecting subject, electronic device, and computer readable storage medium
CN107993209B (en) Image processing method, image processing device, computer-readable storage medium and electronic equipment
CN109242794B (en) Image processing method, image processing device, electronic equipment and computer readable storage medium
CN107862658B (en) Image processing method, image processing device, computer-readable storage medium and electronic equipment
CN107481186B (en) Image processing method, image processing device, computer-readable storage medium and computer equipment
WO2019037739A1 (en) Image processing parameter acquisition method, readable storage medium and computer device
WO2019105304A1 (en) Image white balance processing method, computer readable storage medium, and electronic device
WO2019061659A1 (en) Method and device for removing eyeglasses from facial image, and storage medium
CN107578372B (en) Image processing method, image processing device, computer-readable storage medium and electronic equipment
JP7312026B2 (en) Image processing device, image processing method and program
CN107392870B (en) Image processing method, image processing device, mobile terminal and computer readable storage medium
CN107770446B (en) Image processing method, image processing device, computer-readable storage medium and electronic equipment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18835776

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18835776

Country of ref document: EP

Kind code of ref document: A1