CN113139629A - Font identification method and device, electronic equipment and storage medium - Google Patents
Font identification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113139629A CN113139629A CN202010048253.1A CN202010048253A CN113139629A CN 113139629 A CN113139629 A CN 113139629A CN 202010048253 A CN202010048253 A CN 202010048253A CN 113139629 A CN113139629 A CN 113139629A
- Authority
- CN
- China
- Prior art keywords
- neural network
- network model
- image
- recognized
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000003062 neural network model Methods 0.000 claims abstract description 159
- 238000012545 processing Methods 0.000 claims abstract description 34
- 238000012549 training Methods 0.000 claims description 133
- 238000004891 communication Methods 0.000 claims description 19
- 238000012360 testing method Methods 0.000 claims description 19
- 238000006243 chemical reaction Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 4
- 238000003709 image segmentation Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/24—Character recognition characterised by the processing or recognition method
- G06V30/242—Division of the character sequences into groups prior to recognition; Selection of dictionaries
- G06V30/244—Division of the character sequences into groups prior to recognition; Selection of dictionaries using graphical properties, e.g. alphabet type or font
- G06V30/245—Font recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Character Discrimination (AREA)
Abstract
The embodiment of the invention provides a font identification method, a font identification device, electronic equipment and a storage medium, wherein the method comprises the following steps: inputting a picture to be recognized containing a plurality of characters to be recognized into a first neural network model trained in advance to obtain position coordinates of the characters to be recognized in the picture to be recognized; dividing an image to be recognized based on the position coordinates of a plurality of characters to be recognized to obtain a plurality of single character pictures; respectively converting a plurality of single character pictures into gray level pictures as to-be-processed gray level pictures; respectively carrying out local binarization processing on a plurality of gray level images to be processed to obtain binary images serving as binary images to be input; and respectively inputting the multiple binary images to be input into a pre-trained second neural network model to obtain the fonts of the characters to be recognized contained in each binary image. By adopting the method provided by the embodiment of the invention, the accuracy rate of font identification is improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a font identification method and apparatus, an electronic device, and a storage medium.
Background
The fonts of characters on many pictures are often found interesting in daily life, but the fonts of the characters are difficult to determine because the characters are on the pictures. In order to determine the font of these words, font recognition is very necessary.
The existing identification method mainly comprises the steps of inputting an image containing characters to be identified into a character recognition model, extracting character features of the characters to be identified contained in the input image by the character recognition model, and further comparing the character features with the existing characters to realize character recognition. However, the existing font recognition method is good for the recognition result of the characters contained in the picture with a single picture background, but for the picture with a complex picture background, the character features in the picture become less obvious under the influence of the complex picture background. The complicated picture background can influence the extraction of character features in the picture, so that the recognition result is poor, and the accuracy of font recognition is low.
Disclosure of Invention
An embodiment of the present invention provides a font identification method, apparatus, electronic device and storage medium, so as to solve the problem of low accuracy of font identification.
In order to achieve the above object, an embodiment of the present invention provides a font identification method, including:
inputting a picture to be recognized containing a plurality of characters to be recognized into a pre-trained first neural network model to obtain position coordinates of the characters to be recognized in the picture to be recognized, wherein the first neural network model is obtained by training based on a first training sample set, and the first training sample set comprises: a first sample picture containing first sample characters, and position coordinates of the first sample characters in the first sample picture;
dividing the image to be recognized based on the position coordinates of the characters to be recognized to obtain a plurality of single character pictures, wherein each single character picture comprises one character to be recognized;
respectively converting the single character pictures into gray level pictures as to-be-processed gray level pictures;
respectively carrying out local binarization processing on the multiple gray level images to be processed to obtain binary images serving as binary images to be input;
respectively inputting a plurality of binary images to be input into a pre-trained second neural network model to obtain the fonts of characters to be recognized contained in each binary image, wherein the second neural network model is obtained by training based on a second training sample set, and the second training sample set comprises: a second sample picture containing a second sample text, and a font of the second sample text that is known.
Further, the converting the multiple single character pictures into gray-scale maps respectively as to-be-processed gray-scale maps includes:
converting the single character picture into a gray scale picture aiming at each single character picture;
determining whether the gray scale map meets an image color inversion condition or not for each gray scale map;
when the gray level image meets the image color inversion condition, performing color inversion on the gray level image to obtain an image with the color inverted as a gray level image to be processed; otherwise, the gray-scale image is taken as a gray-scale image to be processed.
Further, the determining whether the gray scale map satisfies an image color inversion condition for each gray scale map includes:
calculating the mean value of pixel values of all pixel points of the gray image as a first mean value aiming at each gray image;
calculating the mean value of the pixel values of the vertex pixels of the gray level image to be used as a second mean value;
and when the first average value is larger than the second average value, determining that the gray-scale image meets the image color inversion condition.
Further, the performing local binarization processing on the multiple to-be-processed gray level maps to obtain binary maps as to-be-input binary maps includes:
performing local binarization processing on each gray level image to be processed to obtain a binary image;
cutting a black area containing the characters to be recognized in the binary image to obtain a circumscribed rectangular image containing the characters to be recognized;
and aiming at each circumscribed histogram, adding a preset number of pixel points around a pixel point matrix of the circumscribed histogram to obtain a new circumscribed histogram serving as a binary image to be input, wherein the pixel values of the preset number of pixel points are all preset pixel values.
Further, the following steps are adopted to obtain the first neural network model based on the training of the first training sample set, and the method includes:
inputting the first sample picture containing the first sample characters contained in the first training sample set into a first to-be-trained neural network model to obtain an output result of the first to-be-trained neural network model;
determining whether a preset training ending condition is met;
if yes, determining the current first to-be-trained neural network model as the trained first neural network model;
and if not, adjusting the first to-be-trained neural network model to obtain a new first to-be-trained neural network model, and starting next training.
Further, training the second neural network model based on a second training sample set by using the following steps, including:
training a second neural network model to be trained for a preset number of times based on the second sample picture containing the second sample characters and the known font of the second sample characters contained in the second training sample set;
calculating the accuracy of a second to-be-trained neural network model by using a test set aiming at the second to-be-trained neural network model after each training, wherein the second to-be-trained neural network model after each training corresponds to one accuracy;
and selecting the trained second neural network model corresponding to the maximum accuracy as the trained second neural network model.
In order to achieve the above object, an embodiment of the present invention further provides a font identification apparatus, including:
the detection module is used for inputting a picture to be recognized containing a plurality of characters to be recognized into a pre-trained first neural network model to obtain position coordinates of the characters to be recognized in the picture to be recognized, wherein the first neural network model is obtained by training based on a first training sample set, and the first training sample set comprises: a first sample picture containing first sample characters, and position coordinates of the first sample characters in the first sample picture;
the image segmentation module is used for segmenting the image to be recognized based on the position coordinates of the characters to be recognized to obtain a plurality of single character pictures, wherein each single character picture comprises one character to be recognized;
the gray level conversion module is used for respectively converting the single character pictures into gray level pictures as to-be-processed gray level pictures;
the binarization module is used for respectively carrying out local binarization processing on the multiple gray level images to be processed to obtain binary images serving as binary images to be input;
the recognition module is configured to input the multiple binary images to be input into a pre-trained second neural network model respectively to obtain a font of a character to be recognized included in each binary image, where the second neural network model is obtained by training based on a second training sample set, and the second training sample set includes: a second sample picture containing a second sample text, and a font of the second sample text that is known.
Further, the gray scale conversion module includes:
the first conversion sub-module is used for converting the single character picture into a gray scale image aiming at each single character picture;
the second conversion submodule is used for determining whether the gray map meets the image color inversion condition or not for each gray map; when the gray level image meets the image color inversion condition, performing color inversion on the gray level image to obtain an image with the color inverted as a gray level image to be processed; otherwise, the gray-scale image is taken as a gray-scale image to be processed.
Further, the second conversion sub-module is specifically configured to calculate, for each of the grayscale images, an average value of pixel values of all pixel points of the grayscale image, and use the average value as a first average value; calculating the mean value of the pixel values of the vertex pixels of the gray level image to be used as a second mean value; and when the first average value is larger than the second average value, determining that the gray-scale image meets the image color inversion condition.
Further, the binarization module is specifically configured to perform local binarization processing on each to-be-processed gray scale map to obtain a binary map; cutting a black area containing the characters to be recognized in the binary image to obtain a circumscribed rectangular image containing the characters to be recognized; and aiming at each circumscribed histogram, adding a preset number of pixel points around a pixel point matrix of the circumscribed histogram to obtain a new circumscribed histogram serving as a binary image to be input, wherein the pixel values of the preset number of pixel points are all preset pixel values.
Further, the apparatus further includes:
the first model training module is used for training based on a first training sample set to obtain the first neural network model by adopting the following steps:
inputting the first sample picture containing the first sample characters contained in the first training sample set into a first to-be-trained neural network model to obtain an output result of the first to-be-trained neural network model;
determining whether a preset training ending condition is met;
if yes, determining the current first to-be-trained neural network model as the trained first neural network model;
and if not, adjusting the first to-be-trained neural network model to obtain a new first to-be-trained neural network model, and starting next training.
Further, the apparatus further includes:
the second model training module is used for training based on a second training sample set to obtain the second neural network model by adopting the following steps:
training a second neural network model to be trained for a preset number of times based on the second sample picture containing the second sample characters and the known font of the second sample characters contained in the second training sample set;
calculating the accuracy of a second to-be-trained neural network model by using a test set aiming at the second to-be-trained neural network model after each training, wherein the second to-be-trained neural network model after each training corresponds to one accuracy;
and selecting the trained second neural network model corresponding to the maximum accuracy as the trained second neural network model.
In order to achieve the above object, an embodiment of the present invention provides an electronic device, which includes a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface are configured to complete communication between the memory and the processor through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing any one of the steps of the font identification method when executing the program stored in the memory.
In order to achieve the above object, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements any of the above-mentioned font identification method steps.
To achieve the above object, an embodiment of the present invention further provides a computer program product containing instructions, which when run on a computer, causes the computer to perform any of the above-mentioned font identification method steps.
The embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides a font identification method, which comprises the steps of inputting a picture to be identified containing a plurality of characters to be identified into a first neural network model trained in advance to obtain position coordinates of the characters to be identified in the picture to be identified; dividing an image to be recognized based on the position coordinates of a plurality of characters to be recognized to obtain a plurality of single character pictures; respectively converting a plurality of single character pictures into gray level pictures as to-be-processed gray level pictures; respectively carrying out local binarization processing on a plurality of gray level images to be processed to obtain binary images serving as binary images to be input; and respectively inputting the multiple binary images to be input into a pre-trained second neural network model to obtain the fonts of the characters to be recognized contained in each binary image. By adopting the method provided by the embodiment of the invention, the position coordinates of the characters to be recognized in the picture to be recognized are determined through the pre-trained first neural network model, and then the picture to be recognized is segmented according to the determined position coordinates of the characters to be recognized to obtain a plurality of single character pictures; local binarization processing is further carried out on the gray level images of the single character images, and background interference of characters to be recognized in the images to be recognized is removed through the first neural network model and the local binarization processing; and further, inputting a plurality of binary images to be input into a pre-trained second neural network model to obtain the fonts of the characters to be recognized contained in each binary image. By combining the neural network model and the local binarization processing, the font identification is realized under the complex picture background, and the accuracy of the font identification is improved.
Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a font identification method according to an embodiment of the present invention;
FIG. 2 is a flow chart of another method for font identification according to an embodiment of the present invention;
FIG. 3 is a diagram of a picture to be recognized according to an embodiment of the present invention;
FIG. 4 is a diagram of a plurality of single character pictures according to an embodiment of the present invention;
FIG. 5 is a flowchart of determining a color inversion condition of an image according to an embodiment of the present invention;
fig. 6 is a flowchart of performing local binarization processing on a grayscale image to be processed according to an embodiment of the present invention;
FIG. 7 is a flowchart of training a first neural network model according to an embodiment of the present invention;
FIG. 8 is a flowchart of training a second neural network model according to an embodiment of the present invention;
fig. 9a is a schematic structural diagram of a font identification apparatus according to an embodiment of the present invention;
fig. 9b is a schematic structural diagram of another font identification apparatus according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention discloses a font identification method, which comprises the following steps as shown in figure 1:
And 102, segmenting an image to be recognized based on the position coordinates of the characters to be recognized to obtain a plurality of single character pictures, wherein each single character picture comprises one character to be recognized.
And 103, converting the multiple single character pictures into gray maps serving as gray maps to be processed respectively.
And 104, respectively carrying out local binarization processing on the multiple gray level images to be processed to obtain binary images serving as binary images to be input.
By adopting the method provided by the embodiment of the invention, the position coordinates of the characters to be recognized in the picture to be recognized are determined through the pre-trained first neural network model, and then the picture to be recognized is segmented according to the determined position coordinates of the characters to be recognized to obtain a plurality of single character pictures; local binarization processing is further carried out on the gray level images of the single character images, and background interference of characters to be recognized in the images to be recognized is removed through the first neural network model and the local binarization processing; and further, inputting a plurality of binary images to be input into a pre-trained second neural network model to obtain the fonts of the characters to be recognized contained in each binary image. By combining the neural network model and the local binarization processing, the font identification is realized under the complex picture background, and the accuracy of the font identification is improved.
The method and apparatus of the present invention will be described in detail with reference to the accompanying drawings using specific embodiments.
In an embodiment of the present invention, as shown in fig. 2, a font identification method provided in an embodiment of the present invention may include the following steps:
step 201, inputting a to-be-recognized picture containing a plurality of to-be-recognized characters into a pre-trained first neural network model to obtain position coordinates of the plurality of to-be-recognized characters in the to-be-recognized picture, wherein the first neural network model is obtained by training based on a first training sample set, and the first training sample set comprises: the first sample picture containing the first sample text, and the position coordinates of the first sample text in the first sample picture.
In one possible embodiment, the picture to be recognized input into the first neural network model may be as shown in fig. 3, and the words to be recognized included in the picture to be recognized as shown in fig. 3 may include "happy", "open", "gate", and "red".
In one possible implementation, the picture to be recognized as shown in fig. 3 may be divided into five single-word pictures, and the divided five single-word pictures may include the words "happy", "open", "gate", and "red" to be recognized, respectively, as shown in fig. 4.
Step 203, converting the single character picture into a gray scale image for each single character picture.
In the embodiment of the present invention, it may be determined, by the method shown in fig. 5, whether each gray scale map satisfies the color inversion condition:
and 502, comparing the first average value with the second average value, and when the first average value is larger than the second average value, determining that the gray-scale image meets the image color inversion condition, otherwise, not meeting the image color inversion condition.
Step 205a, when the gray scale map satisfies the image color inversion condition, color inversion is performed on the gray scale map, and the obtained image after color inversion is used as the gray scale map to be processed, and the process proceeds to step 206.
In this step, for example, if the pixel matrix of the gray-scale image A is
And when the gray level image A meets the image color inversion condition, performing color inversion on the gray level image A to obtain an image after color inversion as a gray level image to be processed, wherein the pixel matrix of the gray level image to be processed is
And step 205b, when the gray-scale map does not meet the image color inversion condition, taking the gray-scale map as a gray-scale map to be processed, and entering step 206.
And step 206, respectively carrying out local binarization processing on the multiple gray level images to be processed to obtain binary images serving as binary images to be input.
In this step, for each binary image, a black area containing the text to be recognized in the binary image may be cut to obtain a circumscribed rectangle image containing the text to be recognized. And aiming at each circumscribed histogram, adding a preset number of pixel points around a pixel point matrix of the circumscribed histogram to obtain a new circumscribed histogram serving as a binary image to be input, wherein the pixel values of the preset number of pixel points are all preset pixel values.
For example, when the pixel matrix of the circumscribed histogram is a 946 × 793 matrix, two additional pixels may be added around the pixel matrix of the circumscribed histogram, that is, two additional [946 × 4+ (793+4) × 4] — 6972 pixels are added, and then the preset number may be set to 6972. The preset pixel value may be 255.
By adopting the method provided by the embodiment of the invention, the position coordinates of the characters to be recognized in the picture to be recognized are determined through the pre-trained first neural network model, and then the picture to be recognized is segmented according to the determined position coordinates of the characters to be recognized to obtain a plurality of single character pictures; then carrying out local binarization processing on the gray level images of the single character images, and cutting black areas of characters to be recognized in the binary image subjected to the local binarization processing through a first neural network model and the local binarization processing to remove background interference of the characters to be recognized in the image to be recognized; and further, inputting a plurality of binary images to be input into a pre-trained second neural network model to obtain the fonts of the characters to be recognized contained in each binary image. By combining the neural network model and the local binarization processing, the font identification is realized under the complex picture background, and the accuracy of the font identification is improved. Moreover, the accuracy of font identification for dark-color and light-color characters can be improved by performing color inversion processing on the gray-scale image satisfying the image color inversion condition.
In the embodiment of the present invention, the local binarization processing may be performed on a plurality of to-be-processed gray-scale maps by using a method as shown in fig. 6, where the method includes:
In this step, the size of the divided region may be divided according to specific situations, for example, the to-be-processed gray-scale map is divided into 10 regions with the same area.
Step 602, for each region, calculating an average value of pixel values of all pixel points in the region as a third average value.
In step 604, when the pixel value of the pixel point is smaller than the third mean value, the pixel value of the pixel point is determined to be 0.
In practical applications, for example, for the divided area a, if the area a is a rectangular area, an integral of the gray scale image to be processed may be calculated by using an integral function in opencv (Open source Computer Vision Library), and then an average of the pixel values of the area a is calculated by using a formula:
I(x，y)＝s(x2，y2)-s(x1，y2)-s(x2，y1)+s(x1，y1)
wherein I (x, y) represents the mean of the pixel values of the area A, (x, y) represents the coordinates of the center point of the rectangular area A, and x1And y1Coordinate value of upper left corner, x, representing rectangular area A2And y2Represents the coordinate value of the lower right corner of the rectangular area a, and s () represents an integral graph.
In an embodiment of the present invention, in a possible implementation manner, the training based on the first training sample set is to obtain the first neural network model, as shown in fig. 7, the method specifically includes the following steps:
step 701, inputting a first sample picture containing first sample characters included in a first training sample set into a first to-be-trained neural network model to obtain an output result of the first to-be-trained neural network model.
In this step, the first training sample set may include a plurality of first sample pictures, where the position coordinates of the text in each first sample picture are marked.
In this step, the first sample picture is input into the first neural network model to be trained, and the parameters of the first neural network model to be trained are adjusted according to the position coordinates of the characters in each first sample picture.
In a specific application, the first neural network model to be trained may be an EAST (Efficient and Scene Text Detector) detection network.
In this step, the preset training termination condition may include:
training the neural network model to be trained for preset times by using the first training sample set, wherein the preset times can be specifically set according to practical application; or,
and inputting a test sample picture contained in the test sample set into the current neural network model to be trained, wherein the numerical value of the calculated loss function is smaller than a preset threshold value, and the preset threshold value can be specifically set according to practical application.
And 703, if yes, determining the current first neural network model to be trained as the trained first neural network model.
Step 704, if not, adjusting the first neural network model to be trained to obtain a new first neural network model to be trained, and returning to step 701 to start the next training.
In this step, adjusting the first to-be-trained neural network model may include:
carrying out adaptive adjustment on parameters of each layer of the first neural network model to be trained;
the model structure of the first neural network model to be trained is adaptively adjusted, for example, the first neural network model to be trained may be subjected to parameter layer addition or subtraction according to the current training result, or the first neural network model to be trained may be subjected to neural network node addition or subtraction according to the current training result.
In an embodiment of the present invention, in a possible implementation manner, a second neural network model is obtained based on training of a second training sample set, as shown in fig. 8, the method may specifically include the following steps:
step 801, training the second neural network model to be trained for a preset number of times based on a second sample picture containing second sample characters and included in the second training sample set and a known font of the second sample characters.
In this step, each second sample picture contains a second sample character, and the second sample picture is a picture subjected to binarization processing.
In a specific application, the second neural Network model to be trained may be a Residual Network model, for example, a ResNet50(Residual Network 50, Residual Network) model, which specifically may be: the dimension of the input layer of ResNet50 is modified to (128, 128, 1), the head full connection of ResNet50 is removed, two full connection layers are changed, a Dropout (exit) mechanism is added between the two full connection layers, the parameter is set to be 0.5, the parameter which indicates that 50% of the first full connection layer is randomly masked during training, and the output dimension of the last full connection layer is consistent with the number of the types of font identification.
In this step, the preset number of times may be specifically set according to the actual application, for example, when the preset number of times is set to 30000 times. And inputting a second sample picture into a second neural network model to be trained in each training, and adjusting the second neural network model to be trained according to the known font of the second sample characters to obtain the trained second neural network model.
Step 802, for the second neural network model to be trained after each training, calculating the accuracy of the second neural network model to be trained by using the test set, wherein the second neural network model to be trained after each training corresponds to one accuracy.
The test set includes: the method comprises the following steps that a plurality of test pictures are obtained, each test picture comprises a test character, the font of the test character is known, and the number of the test pictures can be determined according to the actual application condition.
In this step, when the number of the test pictures is 200, for the second to-be-trained neural network model after each training, 200 test pictures are respectively input into the second to-be-trained neural network model after the training, and the accuracy rate of recognizing the font of the test characters in 200 test pictures is calculated.
And 803, selecting the trained second neural network model corresponding to the maximum accuracy as the trained second neural network model.
In this step, for example, the accuracy of the second neural network model to be trained obtained after each training is calculated by using the test set can be expressed as { A }1，A2，…，AnIn which A1，A2，…，AnAnd respectively representing the accuracy of the second neural network model to be trained obtained after each training, wherein n represents the training times. The maximum accuracy max { A } can be chosen1，A2，…，AnAnd the corresponding trained second neural network model to be trained is used as the trained second neural network model.
Based on the same inventive concept, according to the font identification method provided in the above embodiment of the present invention, correspondingly, another embodiment of the present invention further provides a font identification device, a schematic structural diagram of which is shown in fig. 9a, and specifically includes:
the detection module 901 is configured to input a to-be-recognized picture including a plurality of to-be-recognized characters into a pre-trained first neural network model, so as to obtain position coordinates of the plurality of to-be-recognized characters in the to-be-recognized picture, where the first neural network model is obtained by training based on a first training sample set, and the first training sample set includes: a first sample picture containing first sample characters, and position coordinates of the first sample characters in the first sample picture;
the image segmentation module 902 is configured to segment an image to be recognized based on position coordinates of a plurality of characters to be recognized to obtain a plurality of single character pictures, where each single character picture includes one character to be recognized;
a gray level conversion module 903, configured to convert the multiple single character pictures into gray level images respectively, where the gray level images serve as gray level images to be processed;
a binarization module 904, configured to perform local binarization on the multiple to-be-processed grayscale images respectively to obtain binary images, which are to be input as the to-be-input binary images;
the recognition module 905 is configured to input a plurality of binary images to be input into a pre-trained second neural network model respectively, to obtain a font of a character to be recognized, where each binary image includes the font, the second neural network model is obtained by training based on a second training sample set, and the second training sample set includes: a second sample picture containing a second sample text, and a font of the second sample text that is known.
By adopting the device provided by the embodiment of the invention, the position coordinates of the characters to be recognized in the picture to be recognized are determined through the pre-trained first neural network model, and then the picture to be recognized is segmented according to the determined position coordinates of the characters to be recognized to obtain a plurality of single character pictures; local binarization processing is further carried out on the gray level images of the single character images, and background interference of characters to be recognized in the images to be recognized is removed through the first neural network model and the local binarization processing; and further, inputting a plurality of binary images to be input into a pre-trained second neural network model to obtain the fonts of the characters to be recognized contained in each binary image. By combining the neural network model and the local binarization processing, the font identification is realized under the complex picture background, and the accuracy of the font identification is improved.
Further, as shown in fig. 9b, the grayscale conversion module 903 includes:
a first conversion submodule 906, configured to convert, for each single character picture, the single character picture into a grayscale map;
a second conversion sub-module 907 for determining, for each gray scale map, whether the gray scale map satisfies an image color inversion condition; when the gray level image meets the image color inversion condition, performing color inversion on the gray level image to obtain an image with the color inverted as a gray level image to be processed; otherwise, the gray-scale image is taken as a gray-scale image to be processed.
Further, the second conversion sub-module 907 is specifically configured to calculate, for each gray scale image, an average value of pixel values of all pixel points of the gray scale image, and use the average value as the first average value; calculating the mean value of the pixel values of the vertex pixels of the gray level image to be used as a second mean value; and when the first average value is larger than the second average value, determining that the gray scale map meets the image color inversion condition.
Further, the binarization module 904 is specifically configured to perform local binarization processing on each to-be-processed grayscale image to obtain a binary image; cutting a black area containing characters to be recognized in the binary image aiming at each binary image to obtain a circumscribed rectangular image containing the characters to be recognized; and aiming at each circumscribed histogram, adding a preset number of pixel points around a pixel point matrix of the circumscribed histogram to obtain a new circumscribed histogram serving as a binary image to be input, wherein the pixel values of the preset number of pixel points are all preset pixel values.
Further, as shown in fig. 9b, the font identification apparatus further includes:
a first model training module 908, configured to obtain a first neural network model based on the training of the first training sample set by using the following steps:
inputting a first sample picture containing first sample characters contained in a first training sample set into a first to-be-trained neural network model to obtain an output result of the first to-be-trained neural network model;
determining whether a preset training ending condition is met;
if yes, determining the current first neural network model to be trained as the trained first neural network model;
and if not, adjusting the first to-be-trained neural network model to obtain a new first to-be-trained neural network model, and starting next training.
Further, as shown in fig. 9b, the font identification apparatus further includes:
a second model training module 909, configured to train to obtain a second neural network model based on the second training sample set by using the following steps:
and training the second neural network model to be trained for preset times based on a second sample picture containing second sample characters and a known font of the second sample characters, wherein the second sample picture is contained in the second training sample set.
Calculating the accuracy of a second to-be-trained neural network model by using a test set aiming at the second to-be-trained neural network model after each training, wherein the second to-be-trained neural network model after each training corresponds to one accuracy;
and selecting the trained second neural network model corresponding to the maximum accuracy as the trained second neural network model.
The embodiment of the present invention further provides an electronic device, as shown in fig. 10, which includes a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004, wherein the processor 1001, the communication interface 1002 and the memory 1003 complete mutual communication through the communication bus 1004,
a memory 1003 for storing a computer program;
the processor 1001 is configured to implement the following steps when executing the program stored in the memory 1003:
inputting a picture to be recognized containing a plurality of characters to be recognized into a pre-trained first neural network model to obtain position coordinates of the characters to be recognized in the picture to be recognized, wherein the first neural network model is obtained by training based on a first training sample set, and the first training sample set comprises: a first sample picture containing first sample characters, and position coordinates of the first sample characters in the first sample picture;
dividing the image to be recognized based on the position coordinates of the characters to be recognized to obtain a plurality of single character pictures, wherein each single character picture comprises one character to be recognized;
respectively converting the single character pictures into gray level pictures as to-be-processed gray level pictures;
respectively carrying out local binarization processing on the multiple gray level images to be processed to obtain binary images serving as binary images to be input;
respectively inputting a plurality of binary images to be input into a pre-trained second neural network model to obtain the fonts of characters to be recognized contained in each binary image, wherein the second neural network model is obtained by training based on a second training sample set, and the second training sample set comprises: a second sample picture containing a second sample text, and a font of the second sample text that is known.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In yet another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the font recognition methods described above.
In a further embodiment of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the font recognition methods of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the device, the electronic apparatus and the storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and the relevant points can be referred to the partial description of the method embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (14)
1. A font recognition method, comprising:
inputting a picture to be recognized containing a plurality of characters to be recognized into a pre-trained first neural network model to obtain position coordinates of the characters to be recognized in the picture to be recognized, wherein the first neural network model is obtained by training based on a first training sample set, and the first training sample set comprises: a first sample picture containing first sample characters, and position coordinates of the first sample characters in the first sample picture;
dividing the image to be recognized based on the position coordinates of the characters to be recognized to obtain a plurality of single character pictures, wherein each single character picture comprises one character to be recognized;
respectively converting the single character pictures into gray level pictures as to-be-processed gray level pictures;
respectively carrying out local binarization processing on the multiple gray level images to be processed to obtain binary images serving as binary images to be input;
respectively inputting a plurality of binary images to be input into a pre-trained second neural network model to obtain the fonts of characters to be recognized contained in each binary image, wherein the second neural network model is obtained by training based on a second training sample set, and the second training sample set comprises: a second sample picture containing a second sample text, and a font of the second sample text that is known.
2. The method according to claim 1, wherein the converting the plurality of single character pictures into the gray-scale maps respectively as the gray-scale maps to be processed comprises:
converting the single character picture into a gray scale picture aiming at each single character picture;
determining whether the gray scale map meets an image color inversion condition or not for each gray scale map;
when the gray level image meets the image color inversion condition, performing color inversion on the gray level image to obtain an image with the color inverted as a gray level image to be processed; otherwise, the gray-scale image is taken as a gray-scale image to be processed.
3. The method of claim 2, wherein determining whether the gray map satisfies an image color inversion condition for each gray map comprises:
calculating the mean value of pixel values of all pixel points of the gray image as a first mean value aiming at each gray image;
calculating the mean value of the pixel values of the vertex pixels of the gray level image to be used as a second mean value;
and when the first average value is larger than the second average value, determining that the gray-scale image meets the image color inversion condition.
4. The method according to claim 1, wherein the performing local binarization processing on the plurality of to-be-processed gray-scale maps to obtain binary maps as to-be-input binary maps comprises:
performing local binarization processing on each gray level image to be processed to obtain a binary image;
cutting a black area containing the characters to be recognized in the binary image to obtain a circumscribed rectangular image containing the characters to be recognized;
and aiming at each circumscribed histogram, adding a preset number of pixel points around a pixel point matrix of the circumscribed histogram to obtain a new circumscribed histogram serving as a binary image to be input, wherein the pixel values of the preset number of pixel points are all preset pixel values.
5. The method of any one of claims 1-4, wherein the training of the first neural network model based on the first training sample set comprises:
inputting the first sample picture containing the first sample characters contained in the first training sample set into a first to-be-trained neural network model to obtain an output result of the first to-be-trained neural network model;
determining whether a preset training ending condition is met;
if yes, determining the current first to-be-trained neural network model as the trained first neural network model;
and if not, adjusting the first to-be-trained neural network model to obtain a new first to-be-trained neural network model, and starting next training.
6. The method of any one of claims 1-4, wherein the second neural network model is trained based on a second training sample set using the steps comprising:
training a second neural network model to be trained for a preset number of times based on the second sample picture containing the second sample characters and the known font of the second sample characters contained in the second training sample set;
calculating the accuracy of a second to-be-trained neural network model by using a test set aiming at the second to-be-trained neural network model after each training, wherein the second to-be-trained neural network model after each training corresponds to one accuracy;
and selecting the trained second neural network model corresponding to the maximum accuracy as the trained second neural network model.
7. A font recognition apparatus, comprising:
the detection module is used for inputting a picture to be recognized containing a plurality of characters to be recognized into a pre-trained first neural network model to obtain position coordinates of the characters to be recognized in the picture to be recognized, wherein the first neural network model is obtained by training based on a first training sample set, and the first training sample set comprises: a first sample picture containing first sample characters, and position coordinates of the first sample characters in the first sample picture;
the image segmentation module is used for segmenting the image to be recognized based on the position coordinates of the characters to be recognized to obtain a plurality of single character pictures, wherein each single character picture comprises one character to be recognized;
the gray level conversion module is used for respectively converting the single character pictures into gray level pictures as to-be-processed gray level pictures;
the binarization module is used for respectively carrying out local binarization processing on the multiple gray level images to be processed to obtain binary images serving as binary images to be input;
the recognition module is configured to input the multiple binary images to be input into a pre-trained second neural network model respectively to obtain a font of a character to be recognized included in each binary image, where the second neural network model is obtained by training based on a second training sample set, and the second training sample set includes: a second sample picture containing a second sample text, and a font of the second sample text that is known.
8. The apparatus of claim 7, wherein the gray scale conversion module comprises:
the first conversion sub-module is used for converting the single character picture into a gray scale image aiming at each single character picture;
the second conversion submodule is used for determining whether the gray map meets the image color inversion condition or not for each gray map; when the gray level image meets the image color inversion condition, performing color inversion on the gray level image to obtain an image with the color inverted as a gray level image to be processed; otherwise, the gray-scale image is taken as a gray-scale image to be processed.
9. The apparatus according to claim 8, wherein the second conversion sub-module is specifically configured to calculate, for each of the gray-scale maps, an average of pixel values of all pixel points of the gray-scale map as a first average; calculating the mean value of the pixel values of the vertex pixels of the gray level image to be used as a second mean value; and when the first average value is larger than the second average value, determining that the gray-scale image meets the image color inversion condition.
10. The apparatus according to claim 7, wherein the binarization module is specifically configured to perform local binarization processing on each to-be-processed gray scale map to obtain a binary map; cutting a black area containing the characters to be recognized in the binary image to obtain a circumscribed rectangular image containing the characters to be recognized; and aiming at each circumscribed histogram, adding a preset number of pixel points around a pixel point matrix of the circumscribed histogram to obtain a new circumscribed histogram serving as a binary image to be input, wherein the pixel values of the preset number of pixel points are all preset pixel values.
11. The apparatus of any one of claims 7-10, further comprising:
the first model training module is used for training based on a first training sample set to obtain the first neural network model by adopting the following steps:
inputting the first sample picture containing the first sample characters contained in the first training sample set into a first to-be-trained neural network model to obtain an output result of the first to-be-trained neural network model;
determining whether a preset training ending condition is met;
if yes, determining the current first to-be-trained neural network model as the trained first neural network model;
and if not, adjusting the first to-be-trained neural network model to obtain a new first to-be-trained neural network model, and starting next training.
12. The apparatus of any one of claims 7-10, further comprising:
the second model training module is used for training based on a second training sample set to obtain the second neural network model by adopting the following steps:
training a second neural network model to be trained for a preset number of times based on the second sample picture containing the second sample characters and the known font of the second sample characters contained in the second training sample set;
calculating the accuracy of a second to-be-trained neural network model by using a test set aiming at the second to-be-trained neural network model after each training, wherein the second to-be-trained neural network model after each training corresponds to one accuracy;
and selecting the trained second neural network model corresponding to the maximum accuracy as the trained second neural network model.
13. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 1-6 when executing a program stored in the memory.
14. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010048253.1A CN113139629A (en) | 2020-01-16 | 2020-01-16 | Font identification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010048253.1A CN113139629A (en) | 2020-01-16 | 2020-01-16 | Font identification method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113139629A true CN113139629A (en) | 2021-07-20 |
Family
ID=76808248
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010048253.1A Pending CN113139629A (en) | 2020-01-16 | 2020-01-16 | Font identification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113139629A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113657369A (en) * | 2021-08-26 | 2021-11-16 | 北京有竹居网络技术有限公司 | Character recognition method and related equipment thereof |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6069981A (en) * | 1997-06-16 | 2000-05-30 | Dainippon Screen Mfg. Co., Ltd. | Image conversion method and record medium |
| CN104966097A (en) * | 2015-06-12 | 2015-10-07 | 成都数联铭品科技有限公司 | Complex character recognition method based on deep learning |
| CN109308475A (en) * | 2018-07-26 | 2019-02-05 | 北京百悟科技有限公司 | A kind of character recognition method and device |
| CN109800754A (en) * | 2018-12-06 | 2019-05-24 | 杭州电子科技大学 | A kind of ancient character body classification method based on convolutional neural networks |
-
2020
- 2020-01-16 CN CN202010048253.1A patent/CN113139629A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6069981A (en) * | 1997-06-16 | 2000-05-30 | Dainippon Screen Mfg. Co., Ltd. | Image conversion method and record medium |
| CN104966097A (en) * | 2015-06-12 | 2015-10-07 | 成都数联铭品科技有限公司 | Complex character recognition method based on deep learning |
| CN109308475A (en) * | 2018-07-26 | 2019-02-05 | 北京百悟科技有限公司 | A kind of character recognition method and device |
| CN109800754A (en) * | 2018-12-06 | 2019-05-24 | 杭州电子科技大学 | A kind of ancient character body classification method based on convolutional neural networks |
Non-Patent Citations (2)
| Title |
|---|
| 刘汉英等: "一种新的指纹图像分割算法", 《计算机工程与科学》 * |
| 刘赏等: "《计算机图像和视频处理实验教程》", 电子科技大学出版社 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113657369A (en) * | 2021-08-26 | 2021-11-16 | 北京有竹居网络技术有限公司 | Character recognition method and related equipment thereof |
| CN113657369B (en) * | 2021-08-26 | 2024-05-24 | 北京有竹居网络技术有限公司 | Character recognition method and related equipment thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110176027B (en) | Video target tracking method, device, equipment and storage medium | |
| CN109241861B (en) | Mathematical formula identification method, device, equipment and storage medium | |
| CN111814902A (en) | Target detection model training method, target identification method, device and medium | |
| CN112464809A (en) | Face key point detection method and device, electronic equipment and storage medium | |
| CN110321845B (en) | Method and device for extracting emotion packets from video and electronic equipment | |
| CN110503103B (en) | Character segmentation method in text line based on full convolution neural network | |
| CN110135446B (en) | Text detection method and computer storage medium | |
| CN112101386B (en) | Text detection method, device, computer equipment and storage medium | |
| CN111368632A (en) | Signature identification method and device | |
| CN111833369A (en) | Alum image processing method, system, medium and electronic device | |
| CN112651953A (en) | Image similarity calculation method and device, computer equipment and storage medium | |
| CN110287361B (en) | Figure picture screening method and device | |
| CN113205047A (en) | Drug name identification method and device, computer equipment and storage medium | |
| CN110570442A (en) | Contour detection method under complex background, terminal device and storage medium | |
| CN109389110B (en) | Region determination method and device | |
| CN113971644A (en) | Image identification method and device based on data enhancement strategy selection | |
| CN113221601B (en) | Character recognition method, device and computer readable storage medium | |
| CN110969602B (en) | Image definition detection method and device | |
| CN114841974A (en) | Nondestructive testing method and system for internal structure of fruit, electronic equipment and medium | |
| CN111738272A (en) | Target feature extraction method and device and electronic equipment | |
| CN113139629A (en) | Font identification method and device, electronic equipment and storage medium | |
| CN113887375A (en) | Text recognition method, device, equipment and storage medium | |
| CN113129298A (en) | Definition recognition method of text image | |
| CN117557777A (en) | Sample image determining method and device, electronic equipment and storage medium | |
| CN112712080B (en) | Character recognition processing method for acquiring image by moving character screen |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210720 |
|
| RJ01 | Rejection of invention patent application after publication |