CN109274891B - Image processing method, device and storage medium thereof - Google Patents
Image processing method, device and storage medium thereof Download PDFInfo
- Publication number
- CN109274891B CN109274891B CN201811323983.7A CN201811323983A CN109274891B CN 109274891 B CN109274891 B CN 109274891B CN 201811323983 A CN201811323983 A CN 201811323983A CN 109274891 B CN109274891 B CN 109274891B
- Authority
- CN
- China
- Prior art keywords
- image
- pet face
- processed
- pet
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
- H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
The invention provides an image processing method, an image processing device and a storage medium thereof, and relates to the technical field of image processing. The image processing method comprises the following steps: acquiring an image to be processed, wherein the image to be processed is an image containing a pet face; and positioning the pet face based on a target detection network, and positioning a basic part of the pet face by adopting a characteristic point detection algorithm, wherein the basic part comprises at least one of ears, eyes, a nose, a mouth and a face contour. The image processing method firstly identifies and positions the pet face based on the target detection network, and then positions the basic part of the pet face through the characteristic point detection algorithm, so that the positioning precision of the pet face in the image is improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, and a storage medium thereof.
Background
With the rapid development of computer equipment, networks and image processing technologies, the traditional eye image identification mode has been gradually replaced by an image identification mode automatically performed by a computer, so that the efficiency and the accuracy of image identification are greatly improved. The automatic face recognition through a computer is a common task in the field of computer vision, and is applied in many ways, for example, automatic recognition software on existing mobile terminals such as computers and smart phones can recognize and beautify faces, and more faces appear in daily life and entertainment social interaction of people, for example, light sensation and white degree are added to face areas, characteristic effects are enhanced for local facial features, background pictures are replaced for images, and the like.
Nowadays, people increasingly attach importance to the favorite dogs who are loved by themselves, are willing to share all the favorite dogs, and are keen to enable more people to see the daily and lovely era of pet dogs, but no software capable of accurately recognizing pet faces of pet images exists at present, and the existing recognition and positioning method cannot accurately recognize and position the pet faces.
Disclosure of Invention
In view of the above, an object of the embodiments of the present invention is to provide an image processing method, an image processing apparatus and a storage medium thereof, so as to solve the problem that the conventional identification and positioning method cannot accurately identify and position a pet face.
In a first aspect, an embodiment of the present invention provides an image processing method, where the image processing method includes: acquiring an image to be processed, wherein the image to be processed is an image containing a pet face; and positioning the pet face based on a target detection network, and positioning a basic part of the pet face by adopting a characteristic point detection algorithm, wherein the basic part comprises at least one of ears, eyes, a nose, a mouth and a face contour.
In summary of the first aspect, the acquiring the image to be processed includes: acquiring an image to be processed through a camera; or reading the image to be processed from a local memory; or acquiring the image to be processed on the network through the uniform resource locator.
Synthesize the first aspect, gather the pending image through the camera, include: acquiring a preview video stream acquired by the camera, and identifying whether a pet face exists in the preview video stream based on a target detection network; and if so, taking the image frame of the pet face in the preview video stream as the image to be processed.
In summary of the first aspect, the image processing method further includes: and obtaining an identification frame representing the position of the pet face in the image to be processed based on the positioning result of the target detection network on the pet face. The positioning of the basic part of the pet face by adopting the feature point detection algorithm comprises the following steps: and positioning the basic part of the pet face in the identification frame by adopting a characteristic point detection algorithm.
In summary of the first aspect, the image processing method further includes: responding to a prop attaching instruction, carrying out translation, rotation and zooming operations on the decorative image selected by the user based on the size of the identification frame and the positioning result of the basic part, and dynamically attaching the decorative image after translation, rotation and zooming to the corresponding position of the pet face along with the movement of the characteristic part.
In summary of the first aspect, the image processing method further includes: and storing the image to be processed and the image after the decorative image is added into a cache in a split-image mode by adopting a multi-image-layer storage technology.
In summary of the first aspect, the image processing method further includes: determining the position coordinates of the pet face in the image to be processed based on the positioning result of the target detection network, and determining the feature point coordinates of the pet face based on the positioning result of the basic part; calculating the angle of the pet face based on the feature point coordinates; and acquiring and storing the current image when the light, the position coordinate and the angle of the image to be processed meet the preset conditions.
In summary of the first aspect, the image processing method further includes: performing background segmentation on the pet face based on the basic part to obtain a pet face image of the pet face; and fusing the updated background image and the pet face image to finish background replacement, and obtaining a target image, wherein the updated background image is used as a background image of the target image, and the pet face image is used as a foreground image of the target image.
In a second aspect, an embodiment of the present invention provides an image processing apparatus, including: the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an image to be processed, and the image to be processed is an image containing a pet face; and the positioning module is used for positioning the pet face based on a target detection network and positioning a basic part of the pet face by adopting a characteristic point detection algorithm, wherein the basic part comprises at least one of ears, eyes, a nose, a mouth and a face contour.
In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, where computer program instructions are stored, and when the computer program instructions are read and executed by a processor, the steps in the method in any aspect are performed.
The beneficial effects provided by the invention are as follows:
the invention provides an image processing method, an image processing device and a storage medium thereof, wherein the image processing method adopts a trained target detection network to identify a pet face, and then adopts a characteristic point detection algorithm to position a basic part of the pet face, so that the positioning accuracy of the pet face and the basic part thereof is improved, and the method has lower background false detection rate and universality, thereby having higher accuracy when pet faces are beautified by continuously adding decorative images, replacing backgrounds and the like to images to be processed.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic flowchart of an image processing method according to a first embodiment of the present invention;
FIG. 2 is a flowchart illustrating a step of identifying a pet face in a preview video stream according to a first embodiment of the present invention;
FIG. 3 is a flowchart illustrating a basic portion positioning step according to a first embodiment of the present invention;
FIG. 4 is a schematic flow chart illustrating a decorative image adding step according to a first embodiment of the present invention;
FIG. 5 is a flowchart illustrating a background replacement step according to a first embodiment of the present invention;
FIG. 6 is a flowchart illustrating a pet face image confirmation procedure according to a first embodiment of the present invention;
fig. 7 is a block diagram of an image processing apparatus 100 according to a second embodiment of the present invention;
fig. 8 is a block diagram of an electronic device 200 applicable to the embodiment of the present application according to a third embodiment of the present invention.
Icon: 100-an image processing apparatus; 110-an obtaining module; 120-a positioning module; 200-an electronic device; 201-a memory; 202-a memory controller; 203-a processor; 204-peripheral interface; 205-input-output unit; 206-an audio unit; 207-display unit.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
First embodiment
The research of the applicant finds that more and more applications of the mobile terminal based on face recognition and face beautification appear in daily life and entertainment social contact of people, but no mature pet face recognition method and application can accurately recognize and position a pet face, the defects that an object is easy to move and moving in the pet image shooting process and is difficult to capture and grab cannot be overcome, the position of the pet face in an image cannot be accurately determined, and a customized prop cannot be attached to the corresponding position of the pet face accurately. In order to solve the above problem, a first embodiment of the present invention provides an image processing method applied to a computer or other processing apparatus.
Referring to fig. 1, fig. 1 is a schematic flow chart of an image processing method according to a first embodiment of the present invention, and the image processing method includes the following specific steps:
step S20: and acquiring an image to be processed, wherein the image to be processed is an image containing a pet face.
The image to be processed may be a picture, video or other format image. Meanwhile, in consideration of the normative and unified requirements of image processing, only the image of the specified type may need to be processed, and the user may specify the image of other format which cannot be processed, so that the image determined by the user can be filtered, and when the image specified by the user is in the preset format which can be processed, the image is taken as the image to be processed.
Step S40: and positioning the pet face based on a target detection network, and positioning a basic part of the pet face by adopting a characteristic point detection algorithm, wherein the basic part comprises at least one of ears, eyes, a nose, a mouth and a face contour.
The target detection network in this embodiment is a feature extraction model established based on a convolutional neural network, and the target detection network can identify a target object in a picture and locate the target object after identifying the target object, and the target object in this embodiment is a pet face.
Further, the target detection network in this embodiment may be obtained based on RCNN (regions with CNN features), Fast-RCNN, SPP-Net (spatial Pyramid Pooling network), YOLO or other target detection algorithms. In consideration of the need of detecting the pet face in real time, the YOLOv3 model may be selected as the target detection network in the embodiment, and the YOLOv3 model has high recognition and positioning speed, low background false detection rate and strong universality, so as to enhance the efficiency and accuracy of pet face recognition and positioning.
In image processing, a feature point refers to a point where the image gradation value changes drastically or a point where the curvature is large on an image edge (i.e., an intersection of two edges). The image feature points play an important role in the image matching algorithm based on the feature points. The image feature points can reflect the essential features of the image and can identify the target object in the image. Matching of images can be completed through matching of feature points. The feature point detection is the most effective way to simplify and express high-dimensional image data, and from a data matrix of one image, no information can be seen, so that key information, some basic elements and the relationship thereof in the image must be extracted according to the data so as to more accurately position the basic part of the pet face. For example, the feature points are the left and right eye corners and the upper and lower orbital vertices, the center points of the left and right eye corners and the upper and lower orbital vertices can be determined as the basic parts of the eyes in the pet face.
The feature point detection algorithm in this embodiment may be a method using laplacian gaussian operator for detection (LOG), a method using Hessian matrix (second order differential) of pixel points and determinant values thereof (DOH), a scale invariant feature transform algorithm (SIFT), an accelerated robust feature (SURF), or other feature point detection algorithms.
The image processing method provided by the embodiment of the invention firstly adopts the target detection network to identify and position the pet face, and then accurately positions the basic part of the pet face through the characteristic point detection algorithm, so as to finish the accurate identification and positioning of the pet face and improve the accuracy of the pet face positioning; and simultaneously, the outlines of ears, faces, noses, mouths, faces and the like which are most likely to be decorated or added with decoration and have obvious characteristics are selected as basic parts, so that the positioning accuracy of the pet faces is further improved.
As an alternative implementation manner, before executing step S20, the method defaults to directly start the camera and display the preview video stream of the live view frame of the camera to the user, and simultaneously displays the selection prompt of local uploading and network acquisition to the user, so that the user can select instant shooting, local reading or network acquisition to perform image acquisition.
Considering that the present embodiment is based on the subject (person) shooting/recording process of the object (pet), the camera activated for image acquisition generally defaults to the camera device connected to the processing apparatus, it should be understood that the camera device may be a camera, a computer eye, a digital video camera, etc. connected to the processing apparatus by wire or wirelessly. Meanwhile, the embodiment also provides a lens switching function for the user to switch the image pickup device for acquiring the image to be processed when the processing apparatus is connected with a plurality of image pickup devices.
Further, in addition to the automatic adjustment of the shooting parameters by the processing device and the application program, the embodiment can also provide a shooting regulation function for the user, so that the user can adjust parameters such as aperture, shutter time, sensitivity and the like when shooting images through the shooting device, and the quality of the images is further improved while the personalized requirements of the user are fully met.
When the user selects a network acquisition mode to acquire the image, searching and acquiring a specified image from the network as an image to be processed according to the uniform resource locator input by the user. The image corresponding to the uniform resource locator may be stored in a cloud database, a data server, or other processing devices connected to the processing device through a network.
When a user needs to acquire an image in a local uploading manner, the image to be processed in this embodiment may be a stored image that is directly extracted from an internal memory of the processing device or an external memory connected to the processing device according to a storage address specified by the user.
As an alternative embodiment, when the image is obtained through the camera in real time, considering that the user needs to shoot when the pet face image is included in the view frame of the camera to ensure that there is a pet face in the image to be processed, pet face recognition may be performed on the preview video stream of the camera when the image to be processed is obtained in step S20. Referring to fig. 2, fig. 2 is a schematic flow chart illustrating a step of recognizing a pet face in a preview video stream according to a first embodiment of the present invention, where the step may specifically be as follows:
step S21: and acquiring a preview video stream acquired by the camera, and identifying whether a pet face exists in the preview video stream based on a target detection network.
Step S22: and if so, taking the image frame of the pet face in the preview video stream as the image to be processed.
As an optional implementation manner for step S40, in order to further improve the accuracy of identifying the basic location, an identification frame representing the pet face position may be obtained through the target detection network, and then the basic location of the image in the identification frame may be located by the feature point detection algorithm. Referring to fig. 3, fig. 3 is a schematic flow chart of a basic location positioning step according to a first embodiment of the present invention, which may include the following steps:
step S41: and obtaining an identification frame representing the position of the pet face in the image to be processed based on the positioning result of the target detection network on the pet face.
Step S42: and positioning the basic part of the pet face in the identification frame by adopting a characteristic point detection algorithm.
It should be understood that the identification frame of the pet face obtained in the above steps may be drawn for display so that the user can accurately determine the position of the pet face in the image to be processed. Further, in the preview video stream image in step S21, a recognition frame of the pet face may be displayed so that the user can quickly grasp the shooting timing when the pet face is located at a proper position on the shooting screen when shooting is performed, thereby improving the quality of the pet image.
After the basic parts of the pet face are accurately positioned, the beautification processing can be carried out on the image based on the positioning result.
Referring to fig. 4, fig. 4 is a schematic flow chart illustrating a decorative image adding step according to a first embodiment of the present invention, which may specifically include the following steps:
step S61: and responding to a prop fitting instruction, and performing translation, rotation and scaling operation on the decorative image selected by the user based on the size of the identification frame and the positioning result of the basic part.
Step S62: and dynamically attaching the decorative image after translation, rotation and scaling to the corresponding position of the pet face along with the movement of the characteristic part.
The decoration image in this embodiment may be a corresponding decoration image recommended by the system according to the feature positions of the ear, eye, nose, mouth, face contour, and the like selected by the user, or may be a decoration image selected by the user. The decorative image in the embodiment may be a hat, glasses, earrings, or the like that are matched with the pet face and its five sense organs, or a scarf that is matched with a basic portion such as a neck near the pet face.
It should be understood that the decorative image may also be a specific decorative image that the user draws and saves himself in order to enhance user autonomy and richness of the decorative image.
In the pet face and basic part positioning process, in the process that the decorative image is automatically attached to the basic part, face or head of the pet face, the embodiment adopts a feature point detection algorithm based on a hierarchical networking thought to perform high-precision positioning on the coordinate position (including five basic parts of ear, face, nose, mouth and face outlines and other pet face key points) of the feature point on the pet face, and the decorative image is added to the pet face, so that the high-precision addition of the decorative image is completed, and the matching degree of the decorative image and the pet face is higher.
Referring to fig. 5, fig. 5 is a schematic flow chart of a background replacement step according to a first embodiment of the present invention, which may specifically include the following steps:
step S81: and carrying out background segmentation on the pet face based on the basic part to obtain a pet face image of the pet face.
Step S82: and fusing the updated background image and the pet face image to finish background replacement, and obtaining a target image, wherein the updated background image is used as a background image of the target image, and the pet face image is used as a foreground image of the target image.
It should be noted that there are many methods for fusing two images, such as: and superposing the RGB values of the pixel points at the corresponding positions of the two images.
Further, before step S81, the present embodiment may also check whether a background segmentation object, i.e., a pet face image, exists in the image to be processed. Referring to fig. 6, fig. 6 is a flowchart illustrating a pet face image confirmation procedure according to a first embodiment of the present invention, which may specifically include the following steps:
step S71: and obtaining corresponding pet face parts based on the basic parts, wherein the pet face parts comprise ear regions, face regions, nose regions, mouth regions and face contour regions.
Step S72: and judging whether the image to be processed contains a complete pet face or not according to the pet face part.
Step S73: and if so, obtaining a pet face image contained in the image to be processed according to the pet face part, and taking the pet face image as a background segmentation object.
According to the embodiment of the invention, the pet face image to be beautified is identified from the image to be processed before the background segmentation processing, and the pet face image is determined to be complete, so that the background segmentation is carried out on the pet face image subsequently, and the accuracy of the background segmentation is improved.
As an optional implementation manner, in addition to controlling the camera to shoot an image according to a shooting instruction triggered by a user, the execution device in this embodiment may further control the camera to shoot in a snapshot mode when it is determined that a pet face exists in the picture and the angle, light ray, and position of the picture are appropriate, so as to automatically obtain an image including the pet face. The image acquisition step in the snapshot mode may specifically be: determining the position coordinates of the pet face in the image to be processed based on the positioning result of the target detection network, and determining the feature point coordinates of the pet face based on the positioning result of the basic part; calculating the angle of the pet face based on the feature point coordinates; and acquiring and storing the current image when the light, the position coordinate and the angle of the image to be processed meet the preset conditions.
Furthermore, after the user triggers a shooting instruction, the camera can perform continuous shooting for many times in a continuous shooting mode so as to ensure that images of easily moving and well-moving objects can be acquired.
The image processing method provided by this embodiment may further include a timing function, and after the camera starts to start collecting the preview video stream, once it is detected that a pet face exists in the preview video stream within a preset time and a shutter is not pressed by human intervention, a prompt signal is triggered to remind the user of a next operation behavior.
As an optional implementation manner, when image saving is performed, a multi-layer saving technology may be used to perform image-layer saving on an image to be processed, an image to which a decoration image is added, or an image after background segmentation. Images of different layers can be extracted and processed independently, for example, an image to be processed is the layer 1, an image to which a scarf decoration image is added is the layer 2, an image to which a hat decoration image is added is the layer 3, and a user can modify the layer 2 when needing to perform adjustment such as re-adding of the scarf decoration image, so that the adding operation efficiency of the decoration image is improved.
It should be understood that the images saved in the above map layer may also be saved in a cache, so as to increase the speed of reading pictures when adding different decoration images, and further increase the adding efficiency of the decoration images and the background segmentation efficiency.
In this embodiment, when a local save instruction triggered by a user is received, whether the local save instruction is received is determined, and if yes, an image corresponding to the local save instruction in the cache is saved to the local memory.
Furthermore, after the image is stored in the local memory, an operation space for managing the image can be provided, so that the user can perform operations such as storage setting, browsing, inquiring and the like on the shot and edited image.
It should be understood that the image processing method provided by the embodiment may further include other picture beautifying functions such as adding a filter, adding a background decoration, and the like, besides adding a decorative image and a background replacement on the pet face.
The image processing method provided by the embodiment of the invention adopts the trained target detection network to identify the pet face, and then adopts the characteristic point detection algorithm to position the basic part of the pet face, so that the positioning accuracy of the pet face and the basic part thereof is improved, and the method has lower background false detection rate and universality, thereby having higher accuracy when the pet face is beautified by continuously adding decorative images, replacing backgrounds and the like to the image to be processed.
Second embodiment
In order to cooperate with the image processing method provided by the first embodiment of the present invention, the second embodiment of the present invention also provides an image processing apparatus 100.
Referring to fig. 7, fig. 7 is a block diagram of an image processing apparatus 100 according to a second embodiment of the present invention.
The image processing apparatus 100 includes an acquisition module 110 and a positioning module 120.
The acquiring module 110 is configured to acquire an image to be processed, where the image to be processed is an image including a pet face.
A positioning module 120, configured to position the pet face based on a target detection network, and to position a basic part of the pet face by using a feature point detection algorithm, where the basic part includes at least one of an ear, an eye, a nose, a mouth, and a face contour.
As an optional implementation manner, the image processing apparatus 100 provided in this embodiment may further include a decorative image adding module, configured to perform, in response to the prop attaching instruction, translation, rotation, and zooming operations on a decorative image selected by the user based on the size of the identification frame and the positioning result of the basic portion, and dynamically attach the translated, rotated, and zoomed decorative image to a corresponding position of the pet face along with the movement of the feature portion.
As an optional implementation manner, the image processing apparatus 100 provided in this embodiment may further include a background replacement module, where the background replacement module includes:
the background segmentation unit is used for carrying out background segmentation on the pet face based on the basic part to obtain a pet face image of the pet face;
and the background replacing unit is used for fusing the updated background image and the pet face image to complete background replacement and obtain a target image, wherein the updated background image is used as the background image of the target image, and the pet face image is used as the foreground image of the target image.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the apparatus described above may refer to the corresponding process in the foregoing method, and will not be described in too much detail herein.
Third embodiment
Referring to fig. 8, fig. 8 is a block diagram of an electronic device 200 applicable to the embodiment of the present application according to a third embodiment of the present invention. The electronic device 200 provided in this embodiment may include the image processing apparatus 100, a memory 201, a storage controller 202, a processor 203, a peripheral interface 204, an input/output unit 205, an audio unit 206, and a display unit 207.
The memory 201, the memory controller 202, the processor 203, the peripheral interface 204, the input/output unit 205, the audio unit 206, and the display unit 207 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The image processing apparatus 100 includes at least one software function module which may be stored in the memory 201 in the form of software or firmware (firmware) or solidified in an Operating System (OS) of the image processing apparatus 100. The processor 203 is configured to execute an executable module stored in the memory 201, such as a software functional module or a computer program included in the image processing apparatus 100.
The Memory 201 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 201 is used for storing a program, the processor 203 executes the program after receiving an execution instruction, and the method executed by the server defined by the flow process disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 203, or implemented by the processor 203.
The processor 203 may be an integrated circuit chip having signal processing capabilities. The Processor 203 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor 203 may be any conventional processor or the like.
The peripheral interface 204 couples various input/output devices to the processor 203 as well as to the memory 201. In some embodiments, the peripheral interface 204, the processor 203, and the memory controller 202 may be implemented in a single chip. In other examples, they may be implemented separately from the individual chips.
The input and output unit 205 is used for providing input data for a user to realize the interaction of the user with the server (or the local terminal). The input/output unit 205 may be, but is not limited to, a mouse, a keyboard, and the like.
The audio unit 206 provides an audio interface to the user, which may include one or more microphones, one or more speakers, and audio circuitry.
The display unit 207 provides an interactive interface (e.g., a user operation interface) between the electronic device 200 and a user or is used to display image data for user reference. In this embodiment, the display unit 207 may be a liquid crystal display or a touch display. In the case of a touch display, the display can be a capacitive touch screen or a resistive touch screen, which supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations from one or more locations on the touch display at the same time, and the sensed touch operations are sent to the processor 203 for calculation and processing.
It is to be understood that the configuration shown in fig. 8 is merely exemplary, and the electronic device 200 may include more or fewer components than shown in fig. 8, or may have a different configuration than shown in fig. 8. The components shown in fig. 8 may be implemented in hardware, software, or a combination thereof.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the apparatus described above may refer to the corresponding process in the foregoing method, and will not be described in too much detail herein.
In summary, embodiments of the present invention provide an image processing method, an image processing apparatus, and a storage medium thereof, in which the image processing method identifies a pet face by using a trained target detection network, and then locates a basic part of the pet face by using a feature point detection algorithm, so that the accuracy of locating the pet face and the basic part thereof is improved, and the method has a lower background false detection rate and a lower universality, so that the method has a higher accuracy when pet faces are beautified by continuously performing decoration image addition, background replacement, and the like on an image to be processed.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (7)
1. An image processing method, characterized in that the image processing method comprises:
acquiring an image to be processed, wherein the image to be processed is an image containing a pet face;
positioning the pet face based on a target detection network, and positioning a basic part of the pet face by adopting a characteristic point detection algorithm, wherein the basic part comprises at least one of ears, eyes, a nose, a mouth and a face contour;
determining the position coordinates of the pet face in the image to be processed based on the positioning result of the target detection network, and determining the feature point coordinates of the pet face based on the positioning result of the basic part;
calculating the angle of the pet face based on the feature point coordinates;
acquiring and storing a current image when the light, the position coordinate and the angle of the image to be processed meet preset conditions;
the method further comprises the following steps:
obtaining an identification frame representing the position of the pet face in the image to be processed based on the positioning result of the target detection network on the pet face;
the positioning of the basic part of the pet face by adopting the feature point detection algorithm comprises the following steps:
positioning the basic part of the pet face in the identification frame by adopting a characteristic point detection algorithm;
the method further comprises the following steps:
responding to a prop attaching instruction, carrying out translation, rotation and zooming operations on the decorative image selected by the user based on the size of the identification frame and the positioning result of the basic part, and dynamically attaching the decorative image after translation, rotation and zooming to the corresponding position of the pet face along with the movement of the characteristic part.
2. The image processing method according to claim 1, wherein the acquiring the image to be processed comprises:
acquiring an image to be processed through a camera; or
Reading an image to be processed from a local memory; or
And acquiring the image to be processed on the network through the uniform resource locator.
3. The image processing method according to claim 2, wherein the acquiring the image to be processed by the camera comprises:
acquiring a preview video stream acquired by the camera, and identifying whether a pet face exists in the preview video stream based on a target detection network;
and if so, taking the image frame of the pet face in the preview video stream as the image to be processed.
4. The image processing method according to any one of claims 1 to 3, further comprising:
and storing the image to be processed and the image after the decorative image is added into a cache in a split-image mode by adopting a multi-image-layer storage technology.
5. The image processing method according to claim 1, characterized in that the image processing method further comprises:
performing background segmentation on the pet face based on the basic part to obtain a pet face image of the pet face;
and fusing the updated background image and the pet face image to finish background replacement, and obtaining a target image, wherein the updated background image is used as a background image of the target image, and the pet face image is used as a foreground image of the target image.
6. An image processing apparatus characterized by comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an image to be processed, and the image to be processed is an image containing a pet face;
the positioning module is used for positioning the pet face based on a target detection network and positioning a basic part of the pet face by adopting a characteristic point detection algorithm, wherein the basic part comprises at least one of ears, eyes, a nose, a mouth and a face contour; determining the position coordinates of the pet face in the image to be processed based on the positioning result of the target detection network, and determining the feature point coordinates of the pet face based on the positioning result of the basic part; calculating the angle of the pet face based on the feature point coordinates; acquiring and storing a current image when the light, the position coordinate and the angle of the image to be processed meet preset conditions;
the image processing apparatus is further configured to: obtaining an identification frame representing the position of the pet face in the image to be processed based on the positioning result of the target detection network on the pet face;
the positioning module adopts a feature point detection algorithm to position the basic part of the pet face, and comprises the following steps: positioning the basic part of the pet face in the identification frame by adopting a characteristic point detection algorithm;
the image processing apparatus further includes: and the decorative image adding module is used for responding to a prop attaching instruction, carrying out translation, rotation and zooming operations on the decorative image selected by the user based on the size of the identification frame and the positioning result of the basic part, and dynamically attaching the translated, rotated and zoomed decorative image to the corresponding position of the pet face along with the movement of the characteristic part.
7. A computer readable storage medium having stored thereon computer program instructions which, when read and executed by a processor, perform the steps of the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811323983.7A CN109274891B (en) | 2018-11-07 | 2018-11-07 | Image processing method, device and storage medium thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811323983.7A CN109274891B (en) | 2018-11-07 | 2018-11-07 | Image processing method, device and storage medium thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109274891A CN109274891A (en) | 2019-01-25 |
| CN109274891B true CN109274891B (en) | 2021-06-22 |
Family
ID=65191641
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811323983.7A Active CN109274891B (en) | 2018-11-07 | 2018-11-07 | Image processing method, device and storage medium thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109274891B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109886145B (en) * | 2019-01-29 | 2021-09-03 | 浙江泽曦科技有限公司 | Pet face recognition method and system |
| CN111767914A (en) * | 2019-04-01 | 2020-10-13 | 佳能株式会社 | Target object detection device and method, image processing system, and storage medium |
| CN111325132A (en) * | 2020-02-17 | 2020-06-23 | 深圳龙安电力科技有限公司 | Intelligent monitoring system |
| CN111589132A (en) * | 2020-04-26 | 2020-08-28 | 腾讯科技（深圳）有限公司 | Virtual item display method, computer equipment and storage medium |
| CN113469041A (en) * | 2021-06-30 | 2021-10-01 | 北京市商汤科技开发有限公司 | Image processing method and device, computer equipment and storage medium |
| CN113469914B (en) * | 2021-07-08 | 2024-03-19 | 网易（杭州）网络有限公司 | Animal face beautifying method and device, storage medium and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101377815A (en) * | 2007-08-31 | 2009-03-04 | 卡西欧计算机株式会社 | Image pick-up apparatus having a function of recognizing a face and method of controlling the apparatus |
| JP2011019013A (en) * | 2009-07-07 | 2011-01-27 | Ricoh Co Ltd | Imaging apparatus, area detection method, and program |
| CN102413282A (en) * | 2011-10-26 | 2012-04-11 | 惠州Tcl移动通信有限公司 | Self-photographing guiding method and device |
| CN104081757A (en) * | 2012-02-06 | 2014-10-01 | 索尼公司 | Image processing apparatus, image processing method, program, and recording medium |
| CN106577350A (en) * | 2016-11-22 | 2017-04-26 | 深圳市沃特沃德股份有限公司 | Method and device for recognizing pet type |
| CN107437051A (en) * | 2016-05-26 | 2017-12-05 | 上海市公安局刑事侦查总队 | Image processing method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5385752B2 (en) * | 2009-10-20 | 2014-01-08 | キヤノン株式会社 | Image recognition apparatus, processing method thereof, and program |
| CN107800964A (en) * | 2017-10-26 | 2018-03-13 | 武汉大学 | It is a kind of that method of the face automatic detection with capturing is realized based on dual camera |
-
2018
- 2018-11-07 CN CN201811323983.7A patent/CN109274891B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101377815A (en) * | 2007-08-31 | 2009-03-04 | 卡西欧计算机株式会社 | Image pick-up apparatus having a function of recognizing a face and method of controlling the apparatus |
| JP2011019013A (en) * | 2009-07-07 | 2011-01-27 | Ricoh Co Ltd | Imaging apparatus, area detection method, and program |
| CN102413282A (en) * | 2011-10-26 | 2012-04-11 | 惠州Tcl移动通信有限公司 | Self-photographing guiding method and device |
| CN104081757A (en) * | 2012-02-06 | 2014-10-01 | 索尼公司 | Image processing apparatus, image processing method, program, and recording medium |
| CN107437051A (en) * | 2016-05-26 | 2017-12-05 | 上海市公安局刑事侦查总队 | Image processing method and device |
| CN106577350A (en) * | 2016-11-22 | 2017-04-26 | 深圳市沃特沃德股份有限公司 | Method and device for recognizing pet type |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109274891A (en) | 2019-01-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109274891B (en) | Image processing method, device and storage medium thereof | |
| CN108062526B (en) | Human body posture estimation method and mobile terminal | |
| US10872416B2 (en) | Object oriented image editing | |
| CN110610453B (en) | Image processing method and device and computer readable storage medium | |
| CN108765278B (en) | Image processing method, mobile terminal and computer readable storage medium | |
| CN110532984B (en) | Key point detection method, gesture recognition method, device and system | |
| KR102173123B1 (en) | Method and apparatus for recognizing object of image in electronic device | |
| US9547908B1 (en) | Feature mask determination for images | |
| CN107784282B (en) | Object attribute identification method, device and system | |
| US9131150B1 (en) | Automatic exposure control and illumination for head tracking | |
| JP6046808B2 (en) | Adaptive threshold processing for image recognition. | |
| US10769718B1 (en) | Method, medium, and system for live preview via machine learning models | |
| CN112926423B (en) | Pinch gesture detection and recognition method, device and system | |
| US11176355B2 (en) | Facial image processing method and apparatus, electronic device and computer readable storage medium | |
| KR20190028349A (en) | Electronic device and method for human segmentation in image | |
| CN111008935B (en) | Face image enhancement method, device, system and storage medium | |
| US9436870B1 (en) | Automatic camera selection for head tracking using exposure control | |
| JP2018133019A (en) | Information processing system, information processing method, and program | |
| KR20240131412A (en) | Product Cards by Augmented Reality Content Creators | |
| KR20240128967A (en) | API that provides product cards | |
| CN111199169A (en) | Image processing method and device | |
| KR20240128068A (en) | Dynamically Presenting Augmented Reality Content Generators | |
| KR102303206B1 (en) | Method and apparatus for recognizing object of image in electronic device | |
| CN108540719A (en) | Method and device for taking picture, computer equipment and storage medium | |
| CN106557766B (en) | Fuzzy character processing method and system and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |