CN108038514A - A kind of method, equipment and computer program product for being used to identify image - Google Patents
A kind of method, equipment and computer program product for being used to identify image Download PDFInfo
- Publication number
- CN108038514A CN108038514A CN201711445312.3A CN201711445312A CN108038514A CN 108038514 A CN108038514 A CN 108038514A CN 201711445312 A CN201711445312 A CN 201711445312A CN 108038514 A CN108038514 A CN 108038514A
- Authority
- CN
- China
- Prior art keywords
- image
- feature point
- descriptor
- searching
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000004590 computer program Methods 0.000 title description 2
- 238000001914 filtration Methods 0.000 claims description 23
- 238000009792 diffusion process Methods 0.000 claims description 21
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 238000000605 extraction Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 10
- 239000000654 additive Substances 0.000 claims description 5
- 230000000996 additive effect Effects 0.000 claims description 5
- 238000012360 testing method Methods 0.000 abstract description 13
- 238000013522 software testing Methods 0.000 abstract description 6
- 238000004364 calculation method Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 238000013507 mapping Methods 0.000 description 7
- 230000008901 benefit Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001035 drying Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/34—Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides a kind of method and apparatus for searching target image.This method includes：Extract the fisrt feature point of the first image and the first descriptor of fisrt feature point；Extract the second feature point of the second image and the second descriptor of second feature point；And according to the fisrt feature point and second feature point, the target image for being not less than specific threshold with the matching degree of described first image is searched in second image.Technical scheme can use in software test, so as to find required target image in complicated image, also, even if image is distorted, deforms, rotates, also relatively accurately image can be identified.Thus the automatization level of software test is greatly increased, reduces the workload of manual testing, improves Efficiency of Software Testing.
Description
Technical Field
The present invention relates to the field of image processing, and more particularly, to techniques for recognizing images.
Background
In software testing, such as in game testing, it is often desirable to test games developed by third parties. The game testing includes simple function testing and game advertisement testing of the game. In the using process, due to the particularity of the game, it is found that many games cannot acquire internal resources (such as a control, a resource ID and the like) (non-native android codes or by using OpenGL and ActiveX), and at present, the main mobile-end automation testing tools basically operate on the basis of acquiring internal control elements. Therefore, the conventional testing framework and tool cannot meet the requirement of the automation test of the project group game.
In addition, in the manual game automation test, the click button position is mostly simulated based on the mode of manually marking the picture position, which requires a lot of manpower, and the game needs to be marked from a new state after being updated.
In order to solve the above problems, most of the prior art is based on a simple template matching algorithm, and such an algorithm cannot be well matched in a scene that a button to be recognized rotates or deforms.
Fig. 1a and 1b show exemplary diagrams of matching according to a template in the prior art. Fig. 1a is a view that needs to be taken, and fig. 1b is a view after template matching is taken. It can be seen that in fig. 1b, the floating window in the upper left corner does not match well. Moreover, as the game is updated, the background color of the buttons changes (for example, new buttons need to be added or the functions of the original buttons change), a plurality of matching results are generated, manual screening is also needed, and a large amount of manpower is consumed.
Disclosure of Invention
Therefore, the invention aims to overcome the defects of complicated picture matching, low automation degree and inaccurate matching in the prior art.
According to a first aspect of the present invention, there is provided a method for searching for a target image, comprising: extracting a first feature point of a first image and a first descriptor of the first feature point; extracting a second feature point of a second image and a second descriptor of the second feature point; and searching a target image with the matching degree with the first image not lower than a specific threshold value in the second image according to the first characteristic point and the second characteristic point.
According to an embodiment of the present invention, further comprising: before extracting a first feature point of a first image and a first descriptor of the first feature point, sharpening and/or increasing contrast of the first image; and/or sharpening and/or increasing contrast of the second image before extracting the second feature points of the second image and the second descriptors of the second feature points.
According to an embodiment of the present invention, further comprising: before extracting a first feature point of a first image and a first descriptor of the first feature point, performing nonlinear diffusion filtering on the first image; and/or performing nonlinear diffusion filtering on the second image before extracting the second feature points of the second image and the second descriptors of the second feature points.
According to one embodiment of the invention, wherein the non-linear diffusion filtering is performed by an additive operator splitting algorithm.
According to one embodiment of the present invention, wherein the first descriptor and the second descriptor are RGB values of the first feature point and the second feature point, respectively.
According to an embodiment of the present invention, further comprising: dividing the second image into a plurality of second sub-images; and searching for a target image in the second image, the matching degree of which with the first image is not lower than a specific threshold value, comprises: and traversing and searching a target image which is not lower than a specific threshold in the matching degree with the first image in each second sub-image.
According to one embodiment of the invention, the target image with the matching degree not lower than a specific threshold value with the first image is searched in the second image through cluster matching or fast approximate nearest neighbor searching.
According to an embodiment of the present invention, searching for a target image in the second image, the matching degree of which with the first image is not lower than a specific threshold, according to the first feature point and the second feature point includes: and carrying out Hessian matrix operation on the first characteristic point and the second characteristic point.
According to an embodiment of the present invention, further comprising: determining a first principal direction of the first image from at least a portion of first feature points of the first image; determining a second principal direction of the second image from at least a portion of second feature points of the second image; and searching a target image with the matching degree with the first image not lower than a specific threshold value in the second image according to the first main direction and the second main direction.
According to an embodiment of the invention, wherein determining the first main direction of the first image comprises: searching all feature points within a range of 360 degrees by taking the first feature point as a center, taking the first radius as a search radius and taking a sector with a first angle as a first search surface, and calculating a first vector superposition value of all feature points in the first search surface; taking the direction with the maximum first vector superposition value as a first main direction; and/or determining a second main direction of the second image comprises: searching all the characteristic points within the range of 360 degrees by taking the second characteristic point as a center, taking the second radius as a search radius and taking a sector with a second angle as a second search plane; calculating second vector superposition values of all feature points in the second search plane; and taking the direction in which the second vector superposition value is maximum as a second main direction.
According to one embodiment of the present invention, wherein the first feature point has a first scale parameter, and the first radius is 6 times the first scale parameter; and/or the second feature point has a second scale parameter, the second radius being 6 times the second scale parameter.
According to an embodiment of the invention, wherein the first angle and/or the second angle is 60 degrees.
According to an embodiment of the present invention, wherein calculating the first vector superposition value of all the feature points in the first search plane includes: calculating first order differentials of all adjacent feature points of the first feature point; performing Gaussian weighting on the first order differential to obtain a first weighted value; calculating a first vector superposition value of all feature points in the first search surface according to the first weighted value; and/or calculating second vector superposition values of all feature points in the second search plane comprises: calculating second first order differentials of all adjacent feature points of the second feature point; performing Gaussian weighting on the second first order differential to obtain a second weighted value; and calculating second vector superposition values of all feature points in the second search plane according to the second weighted values.
According to one embodiment of the invention, the gaussian weighting of the first order differential is performed by: the closer to the first feature point, the higher the weight value; and/or gaussian weighting the second first order differential by: the closer to the second feature point, the higher the weight value.
According to a second aspect of the present invention, there is provided an apparatus for finding a target image, comprising: first extraction means for extracting a first feature point of a first image and a first descriptor of the first feature point; second extraction means for extracting a second feature point of a second image and a second descriptor of the second feature point; and searching for a target image, the matching degree of which with the first image is not lower than a specific threshold value, in the second image according to the first characteristic point and the second characteristic point.
According to an embodiment of the present invention, further comprising: the first processing device is used for sharpening and/or increasing the contrast of the first image before extracting the first feature point of the first image and the first descriptor of the first feature point; and/or second processing means for sharpening and/or increasing the contrast of the second image before extracting the second feature points of the second image and the second descriptors of the second feature points.
According to an embodiment of the present invention, further comprising: first filtering means for performing nonlinear diffusion filtering on a first image before extracting a first feature point of the first image and a first descriptor of the first feature point; and/or second filtering means for performing nonlinear diffusion filtering on the second image before extracting second feature points of the second image and second descriptors of the second feature points.
According to one embodiment of the invention, wherein the non-linear diffusion filtering is performed by an additive operator splitting algorithm.
According to one embodiment of the present invention, wherein the first descriptor and the second descriptor are RGB values of the first feature point and the second feature point, respectively.
According to an embodiment of the present invention, further comprising: dividing means for dividing the second image into a plurality of second sub-images; and the searching means comprises: and traversing and searching means for traversing and searching for a target image in each second sub-image, wherein the matching degree of the target image and the first image is not lower than a specific threshold value.
According to one embodiment of the invention, the target image with the matching degree not lower than a specific threshold value with the first image is searched in the second image through cluster matching or fast approximate nearest neighbor searching.
According to an embodiment of the present invention, the searching means includes: and the matrix operation device is used for carrying out Hessian matrix operation on the first characteristic point and the second characteristic point.
According to an embodiment of the present invention, further comprising: first direction determining means for determining a first principal direction of the first image from at least a part of first feature points of the first image; second direction determining means for determining a second principal direction of the second image from at least a part of second feature points of the second image; and the target searching device is used for searching a target image of which the matching degree with the first image is not lower than a specific threshold value in the second image according to the first main direction and the second main direction.
According to an embodiment of the present invention, wherein the first direction determining means includes: the first searching device is used for searching all the characteristic points within the range of 360 degrees by taking the first characteristic point as a center, taking the first radius as a searching radius and taking a sector with a first angle as a first searching surface; first calculation means for calculating first vector superposition values of all feature points in the first search surface; first main direction determining means for setting a direction in which the first vector addition value is largest as a first main direction; and/or the second direction determining means comprises: a second searching device, configured to search all feature points within a range of 360 degrees by using the second feature point as a center, using a second radius as a search radius, and using a sector with a second angle as a second search plane; second calculation means for calculating second vector superposition values of all feature points in the second search plane; second main direction determination means for setting a direction in which the second vector addition value is largest as a second main direction.
According to one embodiment of the present invention, wherein the first feature point has a first scale parameter, and the first radius is 6 times the first scale parameter; and/or the second feature point has a second scale parameter, the second radius being 6 times the second scale parameter.
According to an embodiment of the invention, wherein the first angle and/or the second angle is 60 degrees.
According to one embodiment of the invention, wherein the first computing device comprises: first differential calculation means for calculating first order differentials of all adjacent feature points of the first feature point; a first weighting device, configured to perform gaussian weighting on the first order differential to obtain a first weighted value; first vector calculation means for calculating first vector superposition values of all feature points in the first search plane from the first weighted value; and/or the second computing device comprises: second differential calculation means for calculating second first order differentials of all adjacent feature points of the second feature point; a second weighting device, configured to perform gaussian weighting on the second first-order differential to obtain a second weighted value; second vector calculation means for calculating second vector superposition values of all feature points in the second search plane from the second weighted value.
According to one embodiment of the invention, the gaussian weighting of the first order differential is performed by: the closer to the first feature point, the higher the weight value; and/or gaussian weighting the second first order differential by: the closer to the second feature point, the higher the weight value.
According to a third aspect of the present invention, there is provided an apparatus for finding a target image, comprising: one or more processors; a memory; a program stored in the memory, which when executed by the one or more processors, causes the processors to perform the method as described above.
According to a fourth aspect of the present invention, there is provided a computer readable storage medium storing a program which, when executed by a processor, causes the processor to perform the method as described above.
The technical scheme of the invention can be used in software test, so that the required target image can be found in the complex image, and the image can be accurately identified even if the image is distorted, deformed, rotated and the like. Therefore, the automation level of software testing is greatly improved, the workload of manual testing is reduced, and the software testing efficiency is improved.
It is to be understood that the teachings of the present invention need not achieve all of the above-described benefits, but rather that specific embodiments may achieve specific technical results, and that other embodiments of the present invention may achieve benefits not mentioned above.
Drawings
The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:
in the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
FIGS. 1a and 1b show exemplary diagrams of matching according to a template in the prior art;
FIG. 2 is a flow chart of a method of finding a target image in software according to the present invention;
3 a-3 f are schematic diagrams illustrating the change in the number of feature points after processing an image in different ways;
4 a-4 c show schematic diagrams for determining the main direction of an image according to an example of the invention;
FIGS. 5a to 5c are schematic diagrams illustrating a reference image, an image to be searched and a matching situation of the reference image and the image to be searched, respectively;
FIG. 6 is a block diagram of an apparatus for finding a target image in software according to the present invention;
FIG. 7 illustrates an apparatus for finding a target image in software according to an embodiment of the present invention; and
FIG. 8 depicts a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
The principles and spirit of the present invention will be described with reference to a number of exemplary embodiments. It is understood that these embodiments are given solely for the purpose of enabling those skilled in the art to better understand and to practice the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
The following detailed description of embodiments of the invention refers to the accompanying drawings.
Fig. 2 is a flowchart of a method for finding a target image according to the present invention. As shown in fig. 2, the method of the present invention comprises: extracting a first feature point of a first image and a first descriptor of the first feature point in operation S210; extracting a second feature point of a second image and a second descriptor of the second feature point in operation S220; and, in operation S230, according to the first feature point and the second feature point, searching for a target image in the second image, the matching degree of which with the first image is not lower than a specific threshold value.
In the above description, the first image and the second image may also be referred to as a reference image and an image to be searched, respectively. The reference image may be an icon, a game character image, an image of a game item, etc. The image to be searched can be a picture of a game, the picture may contain the reference image, and an object of the present invention is to find the reference image in the image to be searched, which is a picture having a certain matching degree with the reference image.
In operations S210 and S220, feature points capable of characterizing the image are first extracted from the corresponding image, each feature point having a corresponding attribute, such as a luminance value, an RGB value, and the like.
It should be understood that S210 and S220 shown in fig. 2 are not in a sequential order, and operation S210 may be performed first, and then operation S220 may be performed, or the operations may be performed in a reverse order or simultaneously.
After extracting the feature points of the two images, the degree of similarity or accuracy between the two images can be calculated from the extracted feature points. These feature points may be matched one by one, some feature points may be matched correctly, and some feature points may be matched incorrectly, so that the proportion of correct feature points in all matched feature points may be calculated. When the proportion reaches a certain degree or a specific threshold value, the matching is considered to be successful, so that the image with higher matching degree with the reference image can be found from the image to be searched and used as the target image.
The number of the feature points affects the accuracy of image matching, the more the matching points are, the higher the accuracy of matching is, and the less the matching points are, the lower the accuracy of matching is. Therefore, it is necessary to increase the number of feature points that can be extracted as much as possible.
According to one embodiment of the invention, before extracting a first feature point of a first image and a first descriptor of the first feature point, the first image is sharpened and/or the contrast is increased; and/or sharpening and/or increasing contrast of the second image before extracting the second feature points of the second image and the second descriptors of the second feature points.
Next, a method of increasing the number of feature points to improve matching accuracy will be described.
Fig. 3a to 3f are schematic diagrams illustrating the number of feature points after the image is processed in different ways, wherein fig. 3a illustrates a situation that the original image is not processed; FIG. 3b shows the artwork after the contrast increasing process; FIG. 3c shows the original image after sharpening; FIG. 3d shows the artwork after the brightness increase process; FIG. 3e shows the original image after the color saturation increasing process; and fig. 3f shows the artwork after sharpness increase and contrast increase processing.
As shown in fig. 3a, the number of extracted feature points is 404 in the case of the original image without any processing. After a 5-fold increase in contrast, the number of feature points was increased to 646 over the unprocessed artwork, by more than 50%, as shown in fig. 3 b. It can be seen that increasing the contrast of the original image helps to improve the extraction of the feature points.
Further, the inventors performed a 5-fold sharpening process on the original, and after the process, the number of feature points slightly increased by about 10% compared to the original without any processing, as shown in fig. 3 c. As can be seen, sharpening the original image can improve the extraction of the feature points, but the improvement is small.
In fig. 3d, the brightness of the image is increased by a factor of 5, in which case the number of feature points is sharply reduced to 127, which is a reduction of about 70%. This makes it clear that increasing the brightness of the original image tends to degrade the feature point extraction.
In fig. 3e, the color saturation of the image is increased by a factor of 5, while a small drop in the number of feature points occurs, as shown at 391. Therefore, increasing the color saturation of the original image does not have a significant positive effect on improving the feature point extraction, and even deteriorates the effect to some extent.
In fig. 3f, the sharpness and contrast of the image are improved by 5 times, respectively, in which case the number of feature points is greatly increased to 766, with an increase of approximately 90%. Therefore, increasing the sharpness and contrast of the original image at the same time helps to greatly improve the effect of feature point extraction.
Sharpening and/or increasing contrast of an image may also be implemented by various existing software or applications, for example, by an ImageEnhance function or function of Python Imaging Library (PIL).
The extraction of the feature points is described in detail below.
According to an embodiment of the present invention, a Scale-invariant feature transform (SIFT) algorithm may be used to extract feature points.
According to another embodiment of the present invention, an accelerated-file feature algorithm (SURF) may also be used to extract feature points.
The SIFT algorithm, SURF algorithm and other feature point retrieval algorithms are mainly based on linear Gaussian pyramids to carry out scale decomposition to eliminate noise and extract feature points. However, the gaussian decomposition is at the expense of local precision, and is easy to cause convenient blurring and detail loss.
According to one embodiment of the invention, before extracting a first feature point of a first image and a first descriptor of the first feature point, the first image is subjected to nonlinear diffusion filtering; and/or performing nonlinear diffusion filtering on the second image before extracting the second feature points of the second image and the second descriptors of the second feature points.
The above-mentioned drawbacks of linear filtering can be solved by non-linear diffusion filtering.
The nonlinear diffusion filtering method is to regard the change of the image brightness (L) on different scales as the divergence (divergence) of a flow function (flow function) in a certain form
Wherein,
x is the abscissa of the pixel point;
y is the ordinate of the pixel point;
l is the brightness;
t is evolution time;
is a Gaussian smoothed image LσA gradient of (a);
g is a gaussian smoothed gradient algorithm, and there are generally three forms:
wherein the function g1Preferential retention of high contrast edges, g2Preferentially reserving regions of greater width, g3The inside of the region can be effectively smoothed while preserving boundary information.
In the present invention, g ═ g is preferably employed2。
In the above algorithm, the parameter k is a contrast factor that controls the diffusion level, and the value of the parameter k is 70% of the histogram of the gradient image ▽ L σ.
The numerical value of the parameter k determines how much edge information can be reserved, and the larger the value of k is, the less edge information is reserved; conversely, the smaller the value of k, the more edge information it retains. In the present invention, k ═ 2 can be used.
The divergence of the gradient can be understood as the laplacian, applied to the image in its discrete form:
Δf＝[f(x+1,y)+f(x-1,y)+f(x,y+1)+f(x,y-1)]-4f(x,y) (2)
the diffusion equation is originally used for heat transfer, is applied to an image and is equivalent to the gray level of the image, namely the temperature, and is filtered from a high gray level area to a low gray level area, so that the drying effect is generated.
According to one method of the present invention, the Non-linear Diffusion equation can be solved based on the Forward Euler Scheme (Forward Euler Scheme).
When the nonlinear diffusion equation is solved based on the forward Euler method, the step length of convergence is short, the time consumption is long, and the calculation complexity is high.
According to an embodiment of the present invention, the nonlinear diffusion filtering may be performed by using an Additive Operator Splitting (AOS) algorithm.
The feature points and the descriptors of the feature points may be extracted by using an AKAZE _ Create () function or function in opencv.
Typically, the first image is a button, logo or smaller picture in a game, and the second image may be a game scene in the game that contains the first image. When searching for the first image in the second image, an image having a certain matching degree with the first image may be searched for in the second image in a traversal manner.
According to one embodiment of the present invention, the second image may be divided into a plurality of second sub-images; and searching for a target image in the second image, the matching degree of which with the first image is not lower than a specific threshold value, comprises: and traversing and searching a target image which is not lower than a specific threshold in the matching degree with the first image in each second sub-image.
By dividing the second image into a plurality of sub-images, the amount of calculation can be significantly reduced, and the efficiency of image searching can be improved.
Various methods may be employed to find the image. According to an embodiment of the present invention, the target image having a degree of matching with the first image not lower than a certain threshold may be searched in the second image by cluster matching (which may be implemented by cv2. knnnmatch) or by fast approximate nearest neighbor search (which may be implemented by cv2.flannbase match).
The image is often distorted, e.g., rotated, distorted, etc. For this reason, it is necessary to eliminate such deformation.
According to an embodiment of the present invention, searching for a target image in the second image, the matching degree of which with the first image is not lower than a specific threshold, according to the first feature point and the second feature point includes: and carrying out Hessian matrix operation on the first characteristic point and the second characteristic point.
The operation process of the hessian matrix is as follows:
the Hessian matrix is calculated as follows:
wherein L isxxIs the second partial derivative in the X direction
Wherein L isyyIs the second partial derivative in the y direction
LxyIs the second derivative of the XY direction
Where σ is a scale parameter σiThe integer value of (a). When searching for the extreme point, each pixel point is compared with all the adjacent points, and when the pixel point is larger than all the adjacent points of the scale domain, the pixel point is the extreme point. In the present invention, the extreme point of the 3X3 matrix can be calculated
Under the Gaussian scale space, can makeConvolving an image with a gaussian kernel having a standard deviation σ, which is equivalent to convolving the image with a duration t ═ σ -2And/2 filtering.
Namely:
wherein sigmaiIs a scale parameter of the feature point.
Further, in order to achieve rotational invariance of the image, it is necessary to determine its principal direction from the local image structure of the feature points. That is, according to one embodiment of the present invention, the method of the present invention further does not include: determining a first principal direction of the first image from at least a portion of first feature points of the first image; determining a second principal direction of the second image from at least a portion of second feature points of the second image; and searching a target image with the matching degree with the first image not lower than a specific threshold value in the second image according to the first main direction and the second main direction.
The way in which the main direction of the image is determined is described below using the first image as an example.
According to an embodiment of the invention, determining the first main direction of the first image comprises: searching all feature points within a range of 360 degrees by taking the first feature point as a center, taking the first radius as a search radius and taking a sector with a first angle as a first search surface, and calculating a first vector superposition value of all feature points in the first search surface; and taking the direction with the maximum first vector superposition value as a first main direction.
The manner of determining the second image principal direction is the same as described above. And will not be described in detail herein.
Fig. 4 a-4 c show schematic diagrams for determining the main direction of an image according to an example of the invention.
As shown in fig. 4 a-4 c, letThe scale parameter of the feature point is sigmaiThen the search radius may be set to 6 σ, for exampleiI.e. the radius of the search circle in fig. 4a to 4c is 6 times the characteristic point scale parameter. Performing Gaussian weighting on first-order differential values Lx and Ly of all adjacent points in the search circle, wherein the closer to the feature point, the higher the weighted value is; and the further away from the feature point, the smaller the weight.
These differential values are considered as a set of points in vector space, which are vector-superimposed over a fan-shaped sliding window of an angle, e.g. 60 °, traversing the entire circular area. The angle at which the longest vector is obtained is the principal direction.
In fig. 4b, the neighbor points are found the least, and thus the resulting direction vector has the smallest value; in fig. 4c, the most neighboring points are found, and therefore the value of the obtained direction vector is the largest; in fig. 4a, however, fewer neighbors are found, and the resulting direction vector has a smaller value than that shown in fig. 4c, but a larger value than that shown in fig. 4 b. The resulting direction in fig. 4c is thus the main direction.
Finally, the target image can be found by mapping.
Fig. 5a to 5c are schematic diagrams respectively showing a reference image, an image to be searched and a matching situation of the reference image and the image to be searched.
In FIG. 5a, there is a single image with a plurality of images; fig. 5b includes the image shown in fig. 5a with a certain distortion, and fig. 5b also includes other images than the image shown in fig. 5a, such as the face, hand, and picture in the background of a person; the mapping process is shown in fig. 5 c. In the mapping, some mappings are correct and some mappings are wrong, and the matching degree of the two images is calculated by calculating the proportion of the correct mapping to the total mapping. And after the matching degree exceeds a certain threshold value, the corresponding image is considered to be found. The threshold value can be set by a person skilled in the art on the basis of practical experience.
The technical scheme of the invention can be used in software test, so that the required target image can be found in the complex image, and the image can be accurately identified even if the image is distorted, deformed, rotated and the like. Therefore, the automation level of software testing is greatly improved, the workload of manual testing is reduced, and the software testing efficiency is improved.
It should be immediately apparent that although the above description is made by taking a software test, such as a game test, as an example, it will be understood by those skilled in the art that the present invention is not limited to the above application, but can be applied to any situation where it is necessary to find a corresponding image in another picture from a reference image, such as face recognition, pattern matching, etc.
Fig. 6 is a block diagram of an apparatus for searching a target image according to the present invention.
As shown in fig. 6, the present invention also provides an apparatus for finding a target image, including: a first extracting device 610, configured to extract a first feature point of a first image and a first descriptor of the first feature point; second extracting means 620, configured to extract a second feature point of a second image and a second descriptor of the second feature point; and a searching device 630, configured to search, according to the first feature point and the second feature point, a target image in the second image, where a matching degree with the first image is not lower than a specific threshold value.
According to an embodiment of the present invention, further comprising: the first processing device is used for sharpening and/or increasing the contrast of the first image before extracting the first feature point of the first image and the first descriptor of the first feature point; and/or second processing means for sharpening and/or increasing the contrast of the second image before extracting the second feature points of the second image and the second descriptors of the second feature points.
According to an embodiment of the present invention, further comprising: first filtering means for performing nonlinear diffusion filtering on a first image before extracting a first feature point of the first image and a first descriptor of the first feature point; and/or second filtering means for performing nonlinear diffusion filtering on the second image before extracting second feature points of the second image and second descriptors of the second feature points.
According to one embodiment of the invention, wherein the non-linear diffusion filtering is performed by an additive operator splitting algorithm.
According to one embodiment of the present invention, wherein the first descriptor and the second descriptor are RGB values of the first feature point and the second feature point, respectively.
According to an embodiment of the present invention, further comprising: dividing means for dividing the second image into a plurality of second sub-images; and the searching means comprises: and traversing and searching means for traversing and searching for a target image in each second sub-image, wherein the matching degree of the target image and the first image is not lower than a specific threshold value.
According to one embodiment of the invention, the target image with the matching degree not lower than a specific threshold value with the first image is searched in the second image through cluster matching or fast approximate nearest neighbor searching.
According to an embodiment of the present invention, the searching means includes: and the matrix operation device is used for carrying out Hessian matrix operation on the first characteristic point and the second characteristic point.
According to an embodiment of the present invention, further comprising: first direction determining means for determining a first principal direction of the first image from at least a part of first feature points of the first image; second direction determining means for determining a second principal direction of the second image from at least a part of second feature points of the second image; and the target searching device is used for searching a target image of which the matching degree with the first image is not lower than a specific threshold value in the second image according to the first main direction and the second main direction.
According to an embodiment of the present invention, wherein the first direction determining means includes: the first searching device is used for searching all the characteristic points within the range of 360 degrees by taking the first characteristic point as a center, taking the first radius as a searching radius and taking a sector with a first angle as a first searching surface; first calculation means for calculating first vector superposition values of all feature points in the first search surface; first main direction determining means for setting a direction in which the first vector addition value is largest as a first main direction; and/or the second direction determining means comprises: a second searching device, configured to search all feature points within a range of 360 degrees by using the second feature point as a center, using a second radius as a search radius, and using a sector with a second angle as a second search plane; second calculation means for calculating second vector superposition values of all feature points in the second search plane; second main direction determination means for setting a direction in which the second vector addition value is largest as a second main direction.
According to one embodiment of the present invention, wherein the first feature point has a first scale parameter, and the first radius is 6 times the first scale parameter; and/or the second feature point has a second scale parameter, the second radius being 6 times the second scale parameter.
According to an embodiment of the invention, wherein the first angle and/or the second angle is 60 degrees.
According to one embodiment of the invention, wherein the first computing device comprises: first differential calculation means for calculating first order differentials of all adjacent feature points of the first feature point; a first weighting device, configured to perform gaussian weighting on the first order differential to obtain a first weighted value; first vector calculation means for calculating first vector superposition values of all feature points in the first search plane from the first weighted value; and/or the second computing device comprises: second differential calculation means for calculating second first order differentials of all adjacent feature points of the second feature point; a second weighting device, configured to perform gaussian weighting on the second first-order differential to obtain a second weighted value; second vector calculation means for calculating second vector superposition values of all feature points in the second search plane from the second weighted value.
According to one embodiment of the invention, the gaussian weighting of the first order differential is performed by: the closer to the first feature point, the higher the weight value; and/or gaussian weighting the second first order differential by: the closer to the second feature point, the higher the weight value.
According to a third aspect of the present invention, there is provided an apparatus for finding a target image, comprising: one or more processors; a memory; a program stored in the memory, which when executed by the one or more processors, causes the processors to perform the method as described above.
According to a fourth aspect of the present invention, there is provided a computer readable storage medium storing a program which, when executed by a processor, causes the processor to perform the method as described above.
Exemplary device
Having described the method and apparatus of exemplary embodiments of the present invention, an apparatus for finding a target image according to another aspect of the present invention is described.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer-readable storage medium. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
In some possible embodiments, the apparatus of the present invention may include at least one or more processors, and at least one memory. Wherein the memory stores a program that, when executed by the processor, causes the processor to perform the various steps described in this specification.
The apparatus 1 for finding a target image according to this embodiment of the present invention is described below with reference to fig. 7. The device 1 shown in fig. 7 is only an example and should not bring any limitations to the functionality and scope of use of the embodiments of the present invention.
As shown in FIG. 7, device 1 may take the form of a general purpose computing device, including but not limited to: at least one processor 10, at least one memory 20, and a bus 60 connecting the various system components.
Bus 60 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures.
The memory 20 may include readable media in the form of volatile memory, such as Random Access Memory (RAM)21 and/or cache memory 22, and may further include Read Only Memory (ROM) 23.
Memory 20 may also include program modules 24, such program modules 24 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The device 1 may also communicate with one or more external devices 2 (e.g. a keyboard, a pointing device, a bluetooth device, etc.) and also with one or more other devices. Such communication may be via an input/output (I/O) interface 40 and displayed on the display unit 30. Also, the device 1 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) through the network adapter 50. As shown, the network adapter 50 communicates with other modules in the device 1 over a bus 60. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the device 1, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Exemplary computer readable storage Medium
In some possible embodiments, aspects of the invention may also be embodied in the form of a computer-readable storage medium comprising program code for causing a processor to perform the above-described method when the program code is executed by the processor.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
As shown in fig. 8, a computer-readable storage medium 3 according to an embodiment of the present invention is described, which can employ a portable compact disc read only memory (CD-ROM) and include program codes, and can be run on a terminal device, such as a personal computer. However, the computer-readable storage medium of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device over any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., over the internet using an internet service provider).
Moreover, while the operations of the method of the invention are depicted in the drawings in a particular order, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (10)
1. A method of finding a target image, comprising:
extracting a first feature point of a first image and a first descriptor of the first feature point;
extracting a second feature point of a second image and a second descriptor of the second feature point; and
and searching a target image with the matching degree with the first image not lower than a specific threshold value in the second image according to the first characteristic point and the second characteristic point.
2. The method of claim 1, further comprising:
before extracting a first feature point of a first image and a first descriptor of the first feature point, sharpening and/or increasing contrast of the first image; and/or
Before extracting a second feature point of a second image and a second descriptor of the second feature point, sharpening and/or increasing contrast of the second image.
3. The method of claim 1, further comprising:
before extracting a first feature point of a first image and a first descriptor of the first feature point, performing nonlinear diffusion filtering on the first image; and/or
Before extracting a second feature point of a second image and a second descriptor of the second feature point, performing nonlinear diffusion filtering on the second image.
4. The method of claim 3, wherein the nonlinear diffusion filtering is performed by an additive operator splitting algorithm.
5. The method of claim 1, further comprising:
dividing the second image into a plurality of second sub-images; and is
Searching for a target image in the second image, the matching degree of which with the first image is not lower than a specific threshold value, comprises: and traversing and searching a target image which is not lower than a specific threshold in the matching degree with the first image in each second sub-image.
6. The method of claim 1, wherein the target image having a degree of match with the first image not below a particular threshold is found in the second image by cluster matching or fast approximate nearest neighbor search.
7. The method of claim 1, wherein, according to the first feature point and the second feature point, finding a target image in the second image whose degree of matching with the first image is not lower than a certain threshold value comprises:
and carrying out Hessian matrix operation on the first characteristic point and the second characteristic point.
8. An apparatus for finding a target image, comprising:
first extraction means for extracting a first feature point of a first image and a first descriptor of the first feature point;
second extraction means for extracting a second feature point of a second image and a second descriptor of the second feature point; and
and the searching device is used for searching the target image of which the matching degree with the first image is not lower than a specific threshold value in the second image according to the first characteristic point and the second characteristic point.
9. An apparatus for searching a target image, comprising:
one or more processors;
a memory;
a program stored in the memory, which when executed by the one or more processors, causes the processors to perform the method of any one of claims 1-7.
10. A computer-readable storage medium storing a program which, when executed by a processor, causes the processor to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711445312.3A CN108038514A (en) | 2017-12-27 | 2017-12-27 | A kind of method, equipment and computer program product for being used to identify image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711445312.3A CN108038514A (en) | 2017-12-27 | 2017-12-27 | A kind of method, equipment and computer program product for being used to identify image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108038514A true CN108038514A (en) | 2018-05-15 |
Family
ID=62098034
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711445312.3A Pending CN108038514A (en) | 2017-12-27 | 2017-12-27 | A kind of method, equipment and computer program product for being used to identify image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108038514A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112559341A (en) * | 2020-12-09 | 2021-03-26 | 上海米哈游天命科技有限公司 | Picture testing method, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140226906A1 (en) * | 2013-02-13 | 2014-08-14 | Samsung Electronics Co., Ltd. | Image matching method and apparatus |
| CN105518709A (en) * | 2015-03-26 | 2016-04-20 | 北京旷视科技有限公司 | Method, system and computer program product for identifying human face |
| CN107180436A (en) * | 2017-04-14 | 2017-09-19 | 西安电子科技大学 | A kind of improved KAZE image matching algorithms |
-
2017
- 2017-12-27 CN CN201711445312.3A patent/CN108038514A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140226906A1 (en) * | 2013-02-13 | 2014-08-14 | Samsung Electronics Co., Ltd. | Image matching method and apparatus |
| CN105518709A (en) * | 2015-03-26 | 2016-04-20 | 北京旷视科技有限公司 | Method, system and computer program product for identifying human face |
| CN107180436A (en) * | 2017-04-14 | 2017-09-19 | 西安电子科技大学 | A kind of improved KAZE image matching algorithms |
Non-Patent Citations (4)
| Title |
|---|
| 廖广军 著: "《公安数字影像处理与分析》", 31 March 2017 * |
| 轩落_翼: "opencv3寻找已知物体", 《CSDN博客：HTTPS://BLOG.CSDN.NET/QQ_23880193/ARTICLE/DETAILS/49970685》 * |
| 隋爱娜 等著: "《数字内容安全技术》", 31 October 2016 * |
| 高攀: "基于KAZA算法的图像检索系统的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112559341A (en) * | 2020-12-09 | 2021-03-26 | 上海米哈游天命科技有限公司 | Picture testing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10803554B2 (en) | Image processing method and device | |
| US10936911B2 (en) | Logo detection | |
| CN110032998B (en) | Method, system, device and storage medium for detecting characters of natural scene picture | |
| US9036905B2 (en) | Training classifiers for deblurring images | |
| CN108564082B (en) | Image processing method, device, server and medium | |
| CN108830780B (en) | Image processing method and device, electronic device and storage medium | |
| CN111275034B (en) | Method, device, equipment and storage medium for extracting text region from image | |
| CN107895377B (en) | Foreground target extraction method, device, equipment and storage medium | |
| Li et al. | Automatic comic page segmentation based on polygon detection | |
| GB2553641B (en) | Techniques for selecting objects in images | |
| US20180253852A1 (en) | Method and device for locating image edge in natural background | |
| CN112101386B (en) | Text detection method, device, computer equipment and storage medium | |
| JP6997369B2 (en) | Programs, ranging methods, and ranging devices | |
| CN111444807B (en) | Target detection method, device, electronic equipment and computer readable medium | |
| CN111160288A (en) | Gesture key point detection method and device, computer equipment and storage medium | |
| CN111062854A (en) | Method, device, terminal and storage medium for detecting watermark | |
| CN116168351B (en) | Inspection method and device for power equipment | |
| CN113850238A (en) | Document detection method and device, electronic equipment and storage medium | |
| AU2011265380B2 (en) | Determining transparent fills based on a reference background colour | |
| CN108960247B (en) | Image significance detection method and device and electronic equipment | |
| CN113888635B (en) | Visual positioning method and related device | |
| RU2633182C1 (en) | Determination of text line orientation | |
| CN111160358A (en) | Image binarization method, device, equipment and medium | |
| CN112465050B (en) | Image template selection method, device, equipment and storage medium | |
| CN110660091A (en) | Image registration processing method and device and photographing correction operation system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180515 |