CN112330597A

CN112330597A - Image difference detection method and device and computer equipment

Info

Publication number: CN112330597A
Application number: CN202011094646.2A
Authority: CN
Inventors: 俞元杰; 刘洋
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2020-10-14
Filing date: 2020-10-14
Publication date: 2021-02-05

Abstract

The invention discloses an image difference detection method, an image difference detection device and computer equipment, which are used for solving the problem of low power inspection efficiency. The method comprises the following steps: determining a standard image and an image to be detected; carrying out ORB (object oriented bounding box) oriented rapid rotation type feature point extraction processing on the standard image and the image to be detected to obtain feature points in the standard image and feature points in the image to be detected; matching the characteristic points in the standard image and the characteristic points in the image to be detected to obtain a processed standard image and a processed image to be detected; performing multi-scale feature extraction processing on the processed standard image and the processed image to be detected to obtain a plurality of scale feature maps respectively corresponding to the standard image and the processed image to be detected, and determining a difference result feature map determined based on the plurality of scale feature maps respectively corresponding to the standard image and the processed image to be detected; and carrying out binarization processing on the difference result characteristic diagram to obtain a difference mask image, and determining a difference detection result based on whether a white region connected domain exists in the difference mask image.

Description

Image difference detection method and device and computer equipment

Technical Field

The invention relates to the technical field of computers, in particular to an image difference detection method and device and computer equipment.

Background

At present, along with the continuous development of industrial automation, corresponding requirements are also put forward for the power inspection work. However, in the prior art, the conventional manual detection method is still adopted to perform power inspection, that is, the power box needs to be manually inspected one by one to check whether abnormal conditions occur. The working mode not only needs to consume more manpower and material resources, but also can cause the condition of missing detection and false detection.

It is thus clear that there is the lower problem of efficiency in electric power inspection among the prior art.

Disclosure of Invention

The embodiment of the invention provides an image difference detection method, an image difference detection device and computer equipment, which are used for solving the technical problem of low power inspection efficiency in the prior art.

According to a first aspect of embodiments of the present invention, there is provided an image difference detection method, the method including:

determining a standard image and an image to be detected, wherein the standard image and the image to be detected are images shot by chassis equipment at the same position in different time periods;

carrying out ORB (object oriented features) oriented rapid rotation type feature point extraction processing on the standard image and the image to be detected to obtain feature points in the standard image and feature points in the image to be detected; matching the characteristic points in the standard image and the characteristic points in the image to be detected to obtain a processed standard image and a processed image to be detected;

performing multi-scale feature extraction processing on the processed standard image and the processed image to be detected to obtain a plurality of scale feature maps respectively corresponding to the processed standard image and the processed image to be detected, and determining a difference result feature map determined based on the plurality of scale feature maps respectively corresponding to the processed standard image and the processed image to be detected;

and carrying out binarization processing on the difference result characteristic diagram to obtain a difference mask image, and determining a difference detection result based on whether a white region connected domain exists in the difference mask image.

In a possible implementation manner, the matching processing is performed on the feature points in the standard image and the feature points in the image to be detected to obtain a processed standard image and a processed image to be detected, and the matching processing includes:

subtracting the vertical coordinates of any two feature points with the same horizontal coordinates in the feature points in the standard image and the to-be-detected image, if the subtracted value corresponding to the first feature point in the to-be-detected image is larger than a first threshold value, determining the first feature point as a first abnormal point, and removing the first abnormal point; and the number of the first and second groups,

subtracting the abscissa of any two feature points with the same ordinate in the feature points in the standard image and the to-be-detected image, if the subtracted value corresponding to a second feature point in the to-be-detected image is larger than a second threshold value, determining the second feature point as a second abnormal point, and removing the second abnormal point;

determining a transformation matrix of the standard image and the image to be detected after abnormal point processing based on a random sampling consistency algorithm, and determining whether each element in the transformation matrix belongs to a threshold range;

and if the image belongs to the threshold range, acquiring a processed standard image and a processed image to be detected based on the coordinates of the vertex corresponding to the transformation matrix.

In a possible implementation manner, performing multi-scale feature extraction processing on the processed standard image and the processed image to be detected to obtain a plurality of scale feature maps respectively corresponding to the processed standard image and the processed image to be detected, and determining a difference result feature map determined based on the plurality of scale feature maps respectively corresponding to the processed standard image and the processed image to be detected includes:

determining a first scale feature map and a second scale feature map corresponding to the processed standard image, and determining a third scale feature map and a fourth scale feature map corresponding to the processed image to be detected;

determining a first absolute value set of difference values of corresponding elements at the same coordinate position in the first scale feature map and the third scale feature map, and determining a first difference map and a second difference map based on the first absolute value set;

determining a second absolute value set of difference values of corresponding elements at the same coordinate position in the second scale feature map and the fourth scale feature map, and determining a third difference map and a third difference map based on the second absolute value set;

carrying out weighted addition processing on elements in the first difference map and the second difference map to obtain a first target difference map; performing weighted addition processing on elements in the third difference map and the fourth difference map to obtain a second target difference map;

and obtaining a difference result characteristic diagram according to the first target difference diagram and the second target difference diagram.

In one possible implementation, obtaining a difference result feature map according to the first target difference map and the second target difference map includes:

respectively carrying out up-sampling processing on the first target difference map and the second target difference map to obtain the first target difference map and the second target difference map with the same scale;

and carrying out weighted addition processing on elements in the first target difference graph and the second target difference graph with the same scale to obtain a difference result characteristic graph.

In one possible implementation, the binarizing the difference result feature map to obtain a difference mask image includes:

carrying out binarization processing on the difference result characteristic diagram to obtain a first difference mask image;

and performing image morphology opening operation processing on the first difference mask image to obtain a difference mask image.

In one possible embodiment, determining the difference detection result based on whether a white region connected component exists in the difference mask image includes:

determining all white area connected domains in the difference mask image;

and determining the minimum bounding rectangle of all the white area connected domains, and determining the difference detection result based on the minimum bounding rectangle.

According to a second aspect of embodiments of the present invention, there is provided an image difference detection apparatus, the apparatus including:

the device comprises a first determining unit, a second determining unit and a judging unit, wherein the first determining unit is used for determining a standard image and an image to be detected, and the standard image and the image to be detected are images shot by chassis equipment at the same position in different time periods;

the first processing unit is used for carrying out ORB (object oriented features) oriented fast rotating type feature point extraction processing on the standard image and the image to be detected to obtain feature points in the standard image and feature points in the image to be detected; matching the characteristic points in the standard image and the characteristic points in the image to be detected to obtain a processed standard image and a processed image to be detected;

the second processing unit is used for performing multi-scale feature extraction processing on the processed standard image and the processed image to be detected to obtain a plurality of scale feature maps respectively corresponding to the processed standard image and the processed image to be detected, and determining a difference result feature map determined based on the plurality of scale feature maps respectively corresponding to the processed standard image and the processed image to be detected;

and the second determining unit is used for carrying out binarization processing on the difference result characteristic diagram to obtain a difference mask image and determining a difference detection result based on whether a white area connected domain exists in the difference mask image or not.

In a possible implementation, the first processing unit is further configured to:

In a possible implementation, the second processing unit is further configured to:

and obtaining a difference result characteristic diagram according to the first target difference diagram and the second target difference diagram. In a possible implementation, the second determining unit is further configured to:

In a possible implementation, the second determining unit is further configured to:

determining all white area connected domains in the difference mask image;

According to a third aspect of embodiments of the present invention, there is provided a computer apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the first aspect of the embodiments of the present invention described above and any of the methods referred to in the first aspect.

According to a fourth aspect of embodiments of the present invention, there is provided a storage medium, wherein instructions of the storage medium, when executed by a processor of a computer device, enable the computer device to perform the first aspect of embodiments of the present invention described above and any of the methods that the first aspect may relate to.

According to a fifth aspect of embodiments of the present invention, there is provided a computer program product, which, when run on a computer device, causes the computer device to perform a method of implementing any one of the above-mentioned first aspect and first aspect of embodiments of the present invention.

The technical scheme provided by the embodiment of the invention at least has the following beneficial effects:

in the embodiment of the invention, a standard image and an image to be detected can be determined, wherein the standard image and the image to be detected are images shot at different time periods for the same-position cabinet type equipment (such as an electric box). Then ORB directional fast rotation type characteristic point extraction processing can be carried out on the standard image and the image to be detected, and characteristic points in the standard image and characteristic points in the image to be detected are obtained; and matching the characteristic points in the standard image and the characteristic points in the image to be detected to obtain a processed standard image and a processed image to be detected. That is to say, in the embodiment of the present invention, the feature points in the standard image and the image to be detected may be matched first, and the processed standard image and the image to be detected may be determined according to the matching result. By the method, other background interferences except for the case equipment in the standard image and the image to be detected can be filtered, and whether the power box has abnormal conditions or not can be detected more accurately.

Further, the processed standard image and the processed image to be detected may be subjected to multi-scale feature extraction processing, so as to obtain a plurality of scale feature maps corresponding to the processed standard image and the processed image to be detected, respectively, and determine a difference result feature map determined based on the plurality of scale feature maps corresponding to each other, specifically, the difference result feature map is used to represent a map including a difference between the image to be detected and the standard image. And then, carrying out binarization processing on the difference result characteristic diagram to obtain a difference mask image, and determining a difference detection result based on whether a white area connected domain exists in the difference mask image, wherein if the white area connected domain exists in the difference mask image, it is determined that the standard image and the image to be detected have a difference.

Therefore, in the embodiment of the invention, the processed standard image and the processed image to be detected can be subjected to multi-scale feature extraction processing, so that a plurality of different scale feature maps respectively corresponding to the processed standard image and the processed image to be detected can be obtained. That is, feature descriptions in the processed standard image and the processed image to be detected at multiple scales can be obtained. Then, a difference result feature map determined based on the plurality of scale feature maps respectively corresponding to the plurality of scale feature maps can be determined, and binarization processing is performed on the difference result feature map to obtain a difference mask image. By the mode, different places of the case type equipment in the standard image and the image to be detected can be clearly displayed, so that the case type equipment can be accurately overhauled. In addition, because the detection is directly carried out according to the determined standard image and the image to be detected, the manual inspection is not needed, and the inspection efficiency is greatly improved.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention and are not to be construed as limiting the invention.

FIG. 1 is a schematic diagram of an application scenario in an embodiment of the present invention;

FIG. 2 is a flowchart illustrating an image difference detection method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of an image including a power box in an embodiment of the invention;

FIG. 4 is a diagram illustrating a feature point matching process according to an embodiment of the present invention;

FIG. 5 is a schematic diagram illustrating matching between a standard image and an image to be detected after outliers are removed in the embodiment of the present invention;

FIG. 6 is a block diagram of an image difference detection apparatus according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a computer device in an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. The embodiments and features of the embodiments of the present invention may be arbitrarily combined with each other without conflict. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.

The terms "first," "second," "third," and "fourth" in the description and claims of the invention and the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the term "comprises" and any variations thereof, which are intended to cover non-exclusive protection. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

As described above, in the prior art, when performing power inspection, it is generally necessary to manually go to the field to actually check whether an abnormal condition exists in a power box or other chassis equipment, and then to handle the abnormal condition. However, in such a manner, not only a large amount of manpower and material resources are required, but also a problem that detection of the power box or other chassis equipment is missed due to manual operation of a maintenance master may exist. Therefore, the technical problem of low power routing inspection efficiency exists in the prior art.

In view of this, the present invention provides an image difference detection method, by which an image including a chassis device acquired at a current time and an image acquired from the same position acquired at a previous time period can be automatically compared with each other to determine whether the chassis device is abnormal, thereby implementing rapid detection of the chassis device.

After the design concept of the embodiment of the present invention is introduced, some simple descriptions are made below on application scenarios to which the technical solution in the embodiment of the present invention is applicable, and it should be noted that the application scenarios described in the embodiment of the present invention are for more clearly describing the technical solution in the embodiment of the present invention, and do not form a limitation on the technical solution provided in the embodiment of the present invention.

In the embodiment of the invention, the technical scheme can be applied to the scenes in which the power box inspection is required, such as the power box inspection scene of a factory, the power box inspection scene of a school, the power box inspection scene of a community and the like, and can also be applied to the inspection scenes of other chassis equipment, and the embodiment of the invention is not limited. For convenience of description, the power box inspection scene is described as an example hereinafter.

In an embodiment of the present invention, please refer to an application scenario schematic diagram shown in fig. 1, where fig. 1 includes two parts, namely, a collection device and a computer device, it should be noted that fig. 1 only illustrates an example in which one collection device and one computer device interact with each other, and in a specific implementation process, a plurality of collection devices may interact with one computer device, or a plurality of collection devices may interact with a plurality of computer devices. It should be noted that the foregoing application scenarios are merely illustrative for the convenience of understanding the spirit and principles of the present invention, and the embodiments of the present invention are not limited in this respect. Rather, embodiments of the present invention may be applied to any scenario where applicable.

In a specific implementation process, the acquisition device and the computer device may be in communication connection through one or more networks. The network may be a wired network or a WIreless network, for example, the WIreless network may be a mobile cellular network, or may be a WIreless-Fidelity (WIFI) network, or may also be other possible networks, which is not limited in this embodiment of the present invention.

In a specific implementation process, the aforementioned acquisition device may be any device capable of acquiring image information of the power box, such as a gunlock, a dome camera, an all-in-one camera, an infrared day and night camera, a high-speed dome camera, a web camera, or a gun-and-ball linked camera, and so on. Specifically, the collection equipment may be deployed in a manner that one collection equipment monitors the deployment of one power box, and certainly, the collection equipment near other power boxes may be used to jointly collect image information of the power box, so as to optimize and save resources as much as possible.

In the embodiment of the invention, the acquisition device can send the acquired image containing the power box to the computer device, and then the computer device performs image difference detection processing on the received image, so as to judge whether the power box has an abnormal condition. Further, after the computer device determines that the power box has an abnormal condition, the computer device may also send a prompt message to the associated management device or a terminal corresponding to the inspection staff, so as to repair the power box in time. It should be noted that, in order to facilitate understanding of the technical solution provided by the present invention, the technical solution provided by the present invention is described hereinafter by taking an interaction between a collection device and a computer device as an example.

To further illustrate the scheme of the image difference detection method provided by the embodiment of the present invention, the following detailed description is made with reference to the accompanying drawings and the specific embodiments. Although embodiments of the present invention provide method steps as shown in the following embodiments or figures, more or fewer steps may be included in the method based on conventional or non-inventive efforts. In steps where no necessary causal relationship exists logically, the order of execution of the steps is not limited to that provided by embodiments of the present invention. The method can be executed in sequence or in parallel according to the method shown in the embodiment or the figures when the method is executed in an actual processing procedure or a device (for example, a parallel processor or an application environment of multi-thread processing).

The method for detecting image differences in the embodiment of the present invention is described below with reference to a flowchart of the method shown in fig. 2, and the steps shown in fig. 2 may be executed by a computer device shown in fig. 1. In an implementation, the computer device may be a server, such as a personal computer, a midrange computer, a cluster of computers, and so forth.

Step 201: and determining a standard image and an image to be detected, wherein the standard image and the image to be detected are images shot by the same-position case equipment in different time periods.

In the embodiment of the invention, the acquisition equipment can send the acquired image containing the power box to the computer equipment, so that the computer equipment can determine the standard image and the image to be detected.

In a specific implementation process, the acquisition device may send the image including the power box to the computer device according to a predetermined period, specifically, the predetermined period may be ten minutes, or one minute, or of course, may also be one hour, which is not limited in the embodiment of the present invention.

In the embodiment of the invention, when the computer device receives the image containing the power box sent by the acquisition device, the image sent at the previous moment can be used as a standard image, and the first image received after the previous moment can be used as an image to be detected. For example, if the capture device sends computer device image 1 at 16: 7:30:00, 9/2020 and computer device image 2 at 16: 8:00:00, 9/2020, the computer device may determine image 1 as the standard image and image 2 as the image to be detected.

In the embodiment of the invention, when the computer device receives the image containing the power box sent by the acquisition device, the historical normal image corresponding to the received image can be searched from the query routing inspection database according to other information, such as the position, the place and the camera view angle information of the current image, the historical normal image can be understood as the image containing the power box and the power box is in a normal state, the historical normal image is used as a standard image, and the received image is determined as the image to be detected.

It should be noted that, in the embodiment of the present invention, the standard image and the image to be detected are opposite, that is, one image may be the standard image or the image to be detected. Moreover, the corresponding standard images are not unique for the same power box. In order to better understand the technical solution provided by the embodiment of the present invention, an example will be described hereinafter in which an image captured in a first time period of a power box at the same position is determined as a standard image, and an image captured in a second time period is determined as an image to be detected, where an end time corresponding to the second time period is after an end time of the first time period.

For example, please refer to fig. 3, fig. 3 is a schematic diagram of a standard image and an image to be detected in an embodiment of the present invention. In fig. 3, the left side shows the image to be detected, and the right side shows the standard image.

Step 202: carrying out ORB (object oriented bounding box) oriented rapid rotation type feature point extraction processing on the standard image and the image to be detected to obtain feature points in the standard image and feature points in the image to be detected; and matching the characteristic points in the standard image and the characteristic points in the image to be detected to obtain a processed standard image and a processed image to be detected.

In the embodiment of the present invention, after the standard image and the image to be detected are determined, ORB (Oriented FAST and Rotated FAST feature point) extraction processing may be performed on the standard image and the image to be detected, so that feature points in the standard image and feature points in the image to be detected may be obtained.

In the embodiment of the present invention, after determining the feature points in the standard image and the feature points in the image to be detected, matching processing may be performed on the feature points, and specifically, the matching processing may be performed on the feature points in the standard image and the feature points in the image to be detected by using, but not limited to, the following steps:

step A: subtracting the vertical coordinates of any two feature points with the same horizontal coordinates in the feature points in the standard image and the to-be-detected image, if the subtracted value corresponding to the first feature point in the to-be-detected image is larger than a first threshold value, determining the first feature point as a first abnormal point, and removing the first abnormal point.

And B: subtracting the abscissa of any two feature points with the same ordinate in the feature points in the standard image and the to-be-detected image, if the subtracted value corresponding to the second feature point in the to-be-detected image is larger than a second threshold value, determining the second feature point as a second abnormal point, and removing the second abnormal point.

In the embodiment of the invention, the characteristic point matching can be carried out on the standard image and the image to be detected based on the Hamming distance algorithm. Specifically, the feature points in the standard image and the feature points in the image to be detected may be subtracted from the ordinate of any two feature points having the same abscissa, if it is determined that the subtracted value corresponding to the first feature point in the image to be detected is greater than the first threshold, the first feature point may be determined as a first outlier, and the first outlier may be rejected, and the feature points in the standard image and the abscissa of any two feature points having the same ordinate in the feature points in the image to be detected may be subtracted, if it is determined that the subtracted value corresponding to the second feature point in the image to be detected is greater than the second threshold, the second feature point may be determined as a second outlier, and the second outlier may be rejected.

In a specific implementation process, the first threshold may be determined according to a middle value of an abscissa corresponding to the feature point in the standard image and the feature point in the image to be detected, the second threshold may be determined according to a middle value of a ordinate corresponding to the feature point in the standard image and the feature point in the image to be detected, and of course, other manners are also possible, which is not limited in the embodiment of the present invention. For example, please refer to fig. 4, fig. 4 is a schematic diagram illustrating a process of feature point matching according to an embodiment of the present invention.

It should be noted that, in the embodiment of the present invention, step a and step B may be performed simultaneously or separately, that is, step a may be performed first, or step B may be performed first, which is not limited in the embodiment of the present invention.

In the embodiment of the present invention, please refer to fig. 5, where fig. 5 is a schematic diagram illustrating a matching between a standard image and an image to be detected after removing outliers in the embodiment of the present invention, and it is obvious that feature points in both the standard image and the image to be detected may correspond to each other one by one.

In the embodiment of the invention, the ORB algorithm is adopted to detect and match the characteristic points of the standard image and the image to be detected, and some bad matching points are removed, so that the matching of the standard image and the image to be detected can be quickly and effectively realized, namely, the influence of background factors in some images is reduced, the great influence on the final difference judgment caused by the tiny change of the shooting angle is avoided, and the judgment accuracy is enhanced.

And C: based on random sampling consensus RANSAC algorithm, determining a transformation matrix of the standard image and the image to be detected after abnormal point processing, and determining whether each element in the transformation matrix belongs to a threshold range.

Step D: and if the image belongs to the threshold range, acquiring the processed standard image and the processed image to be detected based on the coordinates of the vertex corresponding to the transformation matrix.

In the embodiment of the present invention, the transformation matrix of the standard image and the image to be detected after the outlier processing may be calculated based on a Random Sample Consensus (Random Sample Consensus). Further, whether each element in the transformation matrix belongs to the threshold range or not can be determined, if not, the abnormal code is directly reported to the corresponding management equipment, and the image to be detected can be reserved for manual inspection and rechecking.

In the embodiment of the present invention, if it is determined that each element in the transformation matrix is in the threshold range, coordinate point positions of four vertex coordinates of the image to be detected after perspective transformation can be determined according to the transformation matrix, and specifically, the coordinate point positions can be recorded as: left upper point TL (x1, y1), right upper point TR (x2, y2), right lower point BR (x3, y3), left lower point BL (x4, y 4).

In the embodiment of the present invention, the image to be detected may be cropped according to the determined coordinate point position, and a cropping zone Z is determined, specifically, a left boundary point x coordinate Zleft of the cropping zone Z is Max (0, Max (x1, x4)), and a right boundary point x coordinate Zright of the cropping zone Z is Min (W, Min (x2, x3)), where W is an original image width of the image to be detected; the upper boundary point y coordinate Ztop of the cropping zone Z is Max (0, Max (y1, y2)), and the lower boundary point y coordinate Zbottom of the cropping zone Z is Min (H, Min (y4, y3)), where H is the original image height of the image to be detected, so that the accurate position coordinate of the cropping zone Z, that is, the upper left vertex coordinate is (zlaft, Ztop), the width zbug-zlaft, and the height Ztop-Zbottom, can be obtained.

In the embodiment of the present invention, after the accurate position coordinates of the cropping zone Z are determined, the image to be detected and the standard image may be cropped according to the coordinates, so as to obtain the processed standard image and the image to be detected.

Step 203: and performing multi-scale feature extraction processing on the processed standard image and the image to be detected to obtain a plurality of scale feature maps corresponding to the processed standard image and the processed image to be detected respectively.

In the embodiment of the invention, a backbone network Resnet-50 of a target detection model Retianet pre-trained on a COCO data set can be used as a feature extractor to respectively perform multi-scale feature extraction processing on the processed standard image and the image to be detected.

In the embodiment of the invention, the pre-training model, namely the target detection model Retianet, is used for carrying out the data enhancement technology in the training process, specifically, the data enhancement technology comprises means of color disturbance and blurring, namely, the color difference interference caused by illumination and fog and rain factors is effectively reduced, so that the interference factors on the features subjected to the multi-scale feature extraction processing by the model are fewer.

In the embodiment of the present invention, when performing scale feature extraction processing on the processed standard image and the image to be detected, a plurality of convolutional layers may be used to perform feature extraction processing, for example, feature descriptions based on the features output by the convolutional layers 1, 3, and 5 are used. It should be noted that, in the embodiment of the present invention, in order to describe the technical solution of the multi-scale feature extraction process more briefly and clearly, the following description will take the feature extraction process of two scales as an example.

In the embodiment of the present invention, the output characteristics of convolutional layer2 and convolutional layer3 may be selected as feature descriptions, where the size of convolutional layer2 is C1 × H1 × W1, where C1 is used to represent the number of channels, and H1 and W1 are used to represent the height and width, respectively; the size of the convolutional layer3 is C2H 2W 2, wherein C2 is used for representing the number of channels, and H2 and W2 are used for representing the height and the width respectively.

In a specific implementation process, a first scale feature map and a second scale feature map corresponding to the processed standard image can be determined through output features of the convolutional layer2 and the convolutional layer3 and are marked as { Fcur1 and Fcur2}, and a third scale feature map and a fourth scale feature map corresponding to the processed image to be detected can be determined and are marked as { Fref1 and Fref2 }; the extraction scales of the first scale feature map and the third scale feature map are the same, and the extraction scales of the second scale feature map and the fourth scale feature map are the same.

Step 204: and determining a difference result feature map determined based on the plurality of scale feature maps respectively corresponding to the plurality of scale feature maps.

In the embodiment of the present invention, after determining the first scale feature map, the second scale feature map, the third scale feature map, and the fourth scale feature map, the absolute difference set D1 ═ abs (Fcur1-Fref1) of Fcur1 and Fref1 may be calculated, specifically, when performing the calculation, the same-position corresponding elements may be subtracted and the absolute value may be taken, and the final result, i.e., the first absolute difference set D1, is also a depth feature tensor with a size of C1 × H1 × W1. And calculating the absolute difference value set D2 of the third scale feature map and the fourth scale feature map, namely Fcur2 and Fref2 as abs (Fcur2-Fref 2).

In the embodiment of the present invention, the depth feature tensor D1 may be maximized pixel by pixel along the depth feature channel, that is, in the direction of C1, to obtain a single-channel difference map, that is, a first difference map Emax1, where Emax1ij is Max (D1ijk), k is {1,2,3, …, C },1 ═ i < ═ height, and 1 ═ j < ═ width. Likewise, the average values are taken pixel by pixel along the direction C1, and a second difference map Eavg1 is obtained, where Eavg1ij is Mean (D1ijk), k is {1,2,3, …, C },1< ═ i < ═ height, and 1< ═ j < <width. Further, since Emax1 and Eavg1 are both H1 × W1, the difference map at the scale, i.e., the first target difference map E1, is finally obtained by weighted addition of Emax1 and Eavg 1: e1 ═ k1 ═ Emax1+ k2 @ Eavg 1. Specifically, the weight values K1 and K2 may be determined according to actual situations, and may be set to K1-1 and K2-2, for example.

In the embodiment of the present invention, the depth feature tensor D2 may be maximized pixel by pixel along the depth feature channel, that is, in the direction of C2, to obtain a single-channel difference map, that is, a third difference map Emax2, where Emax2ij is Max (D2ijk), k is {1,2,3, …, C },1 ═ i < ═ height, and 1 ═ j < ═ width. Likewise, the average values are taken pixel by pixel along the direction C2, and a single-channel difference map, i.e., a fourth difference map Eavg2 is obtained, where Eavg2ij is Mean (D2ijk), k is {1,2,3, …, C },1< ═ i < ═ height, and 1< ═ j < > width. Further, since Emax2 and Eavg2 are both H2 × W2, the difference map at the scale, i.e., the second target difference map E2, is finally obtained by weighted addition of Emax2 and Eavg 2: e2 ═ k3 ═ Emax2+ k4 @ Eavg 2. Specifically, the weight values K3 and K4 may be determined according to actual situations, and may be set to K3-1 and K4-2, for example. It should be noted that K3 and K1 and K4 and K2 may be set to the same weight value, or may be set to different values, which is not limited in the embodiment of the present invention.

In the embodiment of the present invention, after the first target difference map and the second target difference map are obtained, upsampling processing may be performed on the first target difference map and the second target difference map, and the first target difference map and the second target difference map are upsampled to the same size, so that the first target difference map and the second target difference map with the same scale may be obtained. Further, the elements in the first target difference map and the second target difference map with the same scale may be subjected to weighted addition processing to obtain a difference result feature map. Specifically, the difference result feature map is used to characterize a map including the differences between the image to be detected and the standard image.

In the embodiment of the invention, as convolutional neural network features of multiple scales are fused, that is, deep features pay more attention to semantic information and shallow features can be positioned more accurately, more complete feature information can be obtained by combining deep and shallow layers, that is, a multi-scale feature extraction mode, and two pooling features of maxpoling and avgpoling are combined, specifically, by adopting the mode for obtaining the first target difference map and the second target difference map, a better target difference image can be obtained by adopting the mode, so that the information of the determined difference result feature map is more accurate and complete.

Step 205: and carrying out binarization processing on the difference result characteristic diagram to obtain a difference mask image, and determining a difference detection result based on whether a white region connected domain exists in the difference mask image.

In the embodiment of the present invention, binarization processing may be performed on the difference result feature map to obtain a first difference mask image, and then image morphology opening operation processing may be performed on the first difference mask image to obtain a difference mask image.

In the embodiment of the present invention, binarization processing may be performed on the difference result feature map to obtain a first difference mask image, where the specific first difference mask image only includes pixels whose gray values are 0 (black) or 255 (white), that is, the first difference mask image only includes two colors, namely black and white. Further, the first difference mask image may be subjected to an image morphology opening operation process, so that the contour of the power box in the obtained difference mask image is smoother.

In a specific implementation process, the predetermined structural elements may be used to perform an image morphology opening operation on the first difference mask image, specifically, the predetermined structural elements are used to perform an erosion operation on the average mask image to obtain an erosion result map, and then the predetermined structural elements are used to expand the erosion result map, so as to obtain the difference mask image. Specifically, the predetermined structural element may be a 3 × 3 structure, or may be another structure, which may be determined according to an actual implementation situation, and is not limited in the embodiment of the present invention.

In the embodiment of the present invention, in order to better determine the difference detection result, the difference mask image may be processed to visualize the difference in the difference mask image, all white area connected domains in the difference mask image may be determined, and if it is determined that the white area connected domains are included in the difference mask image, it is determined that the difference exists. Further, a minimum bounding rectangle for all white region connected components may be determined, and a difference detection result may be determined based on the minimum bounding rectangle.

In a specific implementation process, a drawing contour function in OpenCV software, such as a findContours () function, can be used for drawing a minimum bounding rectangle, so that the difference between the power box in the standard image and the power box in the image to be detected can be more accurately determined.

In the embodiment of the invention, after the difference between the power box in the standard image and the power box in the image to be detected is determined, the prompt message can be sent to the management device associated with the computer device, specifically, the prompt message can be a directly output image containing the minimum circumscribed rectangle or a voice prompt message. In addition, the prompt information can be sent to the terminal or the equipment corresponding to the staff who is in charge of patrolling and examining the power box, so that the power box can be overhauled. Certainly, the prompt information may include an identifier of the power box, for example, the power box No. 1, location information of the power box, and information around the power box, for example, environment information, weather status information, or the like, which is not limited in the embodiment of the present invention.

Based on the same inventive concept, the embodiment of the invention provides an image difference detection device, which can realize the corresponding functions of the image difference detection method. The image difference detection means may be a hardware structure, a software module, or a hardware structure plus a software module. The image difference detection device can be realized by a chip system, and the chip system can be formed by a chip and can also comprise the chip and other discrete devices. Referring to fig. 6, the image difference detecting apparatus includes:

a first determining unit 601, configured to determine a standard image and an image to be detected, where the standard image and the image to be detected are images captured at different time periods for chassis-type equipment at the same position;

a first processing unit 602, configured to perform ORB directional fast rotation type feature point extraction processing on the standard image and the image to be detected, so as to obtain feature points in the standard image and feature points in the image to be detected; matching the characteristic points in the standard image and the characteristic points in the image to be detected to obtain a processed standard image and a processed image to be detected;

a second processing unit 603, configured to perform multi-scale feature extraction processing on the processed standard image and the processed image to be detected, obtain multiple scale feature maps corresponding to the processed standard image and the processed image to be detected, respectively, and determine a difference result feature map determined based on the multiple scale feature maps corresponding to the processed standard image and the processed image to be detected;

a second determining unit 604, configured to perform binarization processing on the difference result feature map, obtain a difference mask image, and determine a difference detection result based on whether a white region connected domain exists in the difference mask image.

In a possible implementation, the first processing unit 602 is further configured to:

In a possible implementation, the second processing unit 603 is further configured to:

and obtaining a difference result characteristic diagram according to the first target difference diagram and the second target difference diagram. In a possible implementation, the second processing unit 603 is further configured to:

In a possible implementation, the second determining unit 604 is further configured to:

determining all white area connected domains in the difference mask image;

All relevant contents of each step related to the embodiment of the image difference detection method may be referred to the functional description of the functional module corresponding to the image difference detection apparatus in the embodiment of the present invention, and are not described herein again.

The division of the modules in the embodiments of the present invention is schematic, and only one logical function division is provided, and in actual implementation, there may be another division manner, and in addition, each functional module in each embodiment of the present invention may be integrated in one controller, or may exist alone physically, or two or more modules are integrated in one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.

Based on the same inventive concept, an embodiment of the present invention provides a computer apparatus, please refer to fig. 7, where the computer apparatus includes at least one processor 701 and a memory 702 connected to the at least one processor, a specific connection medium between the processor 701 and the memory 702 is not limited in the embodiment of the present invention, in fig. 7, the processor 701 and the memory 702 are connected through a bus 700 as an example, the bus 700 is represented by a thick line in fig. 7, and a connection manner between other components is only schematically illustrated and not limited. The bus 700 may be divided into an address bus, a data bus, a control bus, etc., and is shown in fig. 7 with only one thick line for ease of illustration, but does not represent only one bus or one type of bus. In addition, the image difference detection apparatus further includes a communication interface 703 for receiving image information.

In the embodiment of the present invention, the memory 702 stores instructions executable by the at least one processor 701, and the at least one processor 701 may execute the steps included in the image difference detection method by executing the instructions stored in the memory 702.

The processor 701 is a control center of the computer device, and may connect various portions of the entire computer device by using various interfaces and lines, and perform various functions and process data of the computer device by operating or executing instructions stored in the memory 702 and calling data stored in the memory 702, thereby performing overall monitoring of the computer device.

Optionally, the processor 701 may include one or more processing units, and the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly handles an operating system, a user interface, an application program, and the like, and the modem processor mainly handles wireless communication. It will be appreciated that the modem processor described above may not be integrated into the processor 701. In some embodiments, processor 701 and memory 702 may be implemented on the same chip, or in some embodiments, they may be implemented separately on separate chips.

The processor 701 may be a general-purpose processor, such as a Central Processing Unit (CPU), digital signal processor, application specific integrated circuit, field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like, that may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present invention. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

Memory 702, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory 702 may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charge Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and so on. The memory 702 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 702 of embodiments of the present invention may also be circuitry or any other device capable of performing a storage function to store program instructions and/or data.

By programming the processor 701, the codes corresponding to the image difference detection method described in the foregoing embodiment may be solidified into a chip, so that the chip can execute the steps of the image difference detection method when running.

Based on the same inventive concept, embodiments of the present invention further provide a storage medium storing computer instructions, which, when executed on a computer, cause the computer to perform the steps of the image difference detection method as described above.

In some possible embodiments, aspects of the image difference detection method provided by the present invention may also be implemented in the form of a program product including program code for causing a control computer device to perform the steps of the image difference detection method according to various exemplary embodiments of the present invention described above in this specification when the program product is run on the control computer device.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims

1. An image difference detection method, characterized in that the method comprises:

2. The method of claim 1, wherein the matching processing is performed on the feature points in the standard image and the feature points in the image to be detected to obtain a processed standard image and a processed image to be detected, and the method comprises:

3. The method of claim 1, wherein performing multi-scale feature extraction processing on the processed standard image and the processed image to be detected to obtain a plurality of scale feature maps corresponding to the processed standard image and the processed image to be detected respectively, and determining a difference result feature map determined based on the plurality of scale feature maps corresponding to each, comprises:

4. The method of claim 3, wherein obtaining a difference result signature from the first target signature and the second target signature comprises:

5. The method according to any one of claims 1 to 4, wherein the binarizing process on the difference result feature map to obtain a difference mask image comprises:

6. The method of claim 5, wherein determining a difference detection result based on whether a white region connected component is present in the difference mask image comprises:

determining all white area connected domains in the difference mask image;

7. An image difference detection apparatus, characterized in that the apparatus comprises:

8. A computer device, characterized in that the computer device comprises: memory, processor and computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the image difference detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, carries out the steps of the image difference detection method according to any one of claims 1 to 6.

10. A computer program product, characterized in that it, when run on a computer device, causes the computer device to carry out the steps of the image difference detection method according to any one of claims 1 to 6.