[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN112017218A - Image registration method and device, electronic equipment and storage medium - Google Patents

Image registration method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN112017218A
CN112017218A CN202010941346.7A CN202010941346A CN112017218A CN 112017218 A CN112017218 A CN 112017218A CN 202010941346 A CN202010941346 A CN 202010941346A CN 112017218 A CN112017218 A CN 112017218A
Authority
CN
China
Prior art keywords
registered
image
target
images
numerical value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010941346.7A
Other languages
Chinese (zh)
Other versions
CN112017218B (en
Inventor
方吉庆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN202010941346.7A priority Critical patent/CN112017218B/en
Publication of CN112017218A publication Critical patent/CN112017218A/en
Application granted granted Critical
Publication of CN112017218B publication Critical patent/CN112017218B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10048Infrared image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30248Vehicle exterior or interior

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The image registration method, the image registration device, the electronic equipment and the storage medium provided by the embodiment of the invention can acquire an image set to be registered; carrying out binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarization images; carrying out connected domain detection on each binary image to obtain one or more connected domains; detecting the target to be registered in each connected domain to obtain the position coordinates of the target to be registered in each connected domain; and registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered to obtain a fused image. Therefore, the coordinates of at least four points of the target to be registered of the multiple images to be registered are obtained, and the images to be registered in the image set to be registered are registered and fused according to the coordinates of the at least four points, so that the fused images with higher definition are obtained, and the details in the images are convenient to identify.

Description

Image registration method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of information technologies, and in particular, to an image registration method and apparatus, an electronic device, and a storage medium.
Background
At present, the collection and snapshot of images through a camera have been widely applied. For example, the traffic camera is used for capturing the vehicle in the running process, so that the violation behaviors can be conveniently monitored and recorded.
However, when the traffic camera is used for collecting and capturing images, situations such as unclear capturing and blurred photos often occur due to the captured objects, weather and the like, so that difficulty is caused in recognizing the captured photos.
Disclosure of Invention
The embodiment of the invention aims to provide an image registration method, an image registration device, electronic equipment and a storage medium, so as to generate a picture image with higher definition through a plurality of pictures. The specific technical scheme is as follows:
in a first aspect of embodiments of the present application, there is provided an image registration method, including:
acquiring an image set to be registered, wherein the image set to be registered comprises at least two images to be registered, and each image to be registered comprises the same target to be registered;
carrying out binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarization images;
carrying out connected domain detection on each binary image to obtain one or more connected domains;
detecting the target to be registered in each connected domain to obtain the position coordinates of the target to be registered in each connected domain, wherein the position coordinates of the target to be registered comprise the coordinates of at least four points in the target to be registered;
and registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered to obtain a fused image.
Optionally, the binarization processing is performed on each image to be registered in the image set to be registered to obtain a plurality of binarized images, including:
carrying out graying processing on each image to be registered in the image set to be registered to obtain a plurality of grayscale images;
respectively calculating the gray level threshold value of each gray level image by a maximum inter-class difference method OTSU;
and aiming at each gray image, setting pixels with the gray values larger than the gray threshold value of the gray image in the gray image as first gray values, and setting pixels with the gray values not larger than the gray threshold value of the gray image in the gray image as second gray values, thereby obtaining a plurality of binary images.
Optionally, performing connected domain detection on each binarized image to obtain one or more connected domains, including:
detecting connected domains of the binary images, and marking a numerical value for each pixel point in the binary images, wherein the numerical values of the pixel points in the same connected domain are equal;
the numerical value of each pixel point in the binary image is re-marked through a preset rule, and the re-marked numerical value of each pixel point is obtained;
and dividing the pixel points with the same numerical value into the same connected domain according to the re-marked numerical value of each pixel point to obtain one or more connected domains.
Optionally, the preset rule includes:
aiming at any pixel point, the numerical value of any pixel point is a first numerical value, and the numerical values of a plurality of pixel points adjacent to any pixel point are all second numerical values;
when the first numerical value is equal to the second numerical value, the numerical value of any pixel point is not marked again;
when the first numerical value is not equal to the second numerical value, the numerical value of any pixel point is marked as the second numerical value again.
Optionally, the preset rule includes:
aiming at any pixel point, the numerical value of any pixel point is a first numerical value, and when the numerical values of all the pixel points are not completely the same in a plurality of pixel points adjacent to any pixel point;
selecting a numerical value with the minimum numerical value of each pixel point from a plurality of adjacent pixel points of any pixel point as a target numerical value;
and marking the numerical value of any pixel point as a target numerical value.
Optionally, the detecting the target to be registered for each connected domain to obtain the position coordinates of the target to be registered for each connected domain includes:
identifying the target to be registered for each connected domain to obtain a binary image of the target to be registered in each binary image;
and projecting the binary image of the target to be registered in each image to be registered to obtain the position coordinates of the target to be registered in each connected domain.
Optionally, registering and fusing each image to be registered in the image set to be registered based on coordinates of at least four points of the target to be registered to obtain a fused image, including:
calculating to obtain a homography matrix between the images to be registered based on the coordinates of at least four points of the target to be registered;
obtaining a mapping relation between the images to be registered according to the homography matrix between the images to be registered;
and registering and fusing the images to be registered in the image set to be registered according to the mapping relation among the images to be registered to obtain fused images.
In a second aspect of the embodiments of the present application, there is provided an image registration apparatus, including:
the image acquisition module is used for acquiring an image set to be registered, wherein the image set to be registered comprises at least two images to be registered, and each image to be registered comprises the same target to be registered;
the binarization processing module is used for carrying out binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarization images;
the connected domain detection module is used for detecting the connected domains of the binary images to obtain one or more connected domains;
the target detection module is used for detecting the target to be registered in each connected domain to obtain the position coordinates of the target to be registered in each connected domain, wherein the position coordinates of the target to be registered comprise the coordinates of at least four points in the target to be registered;
and the image fusion module is used for registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered to obtain a fused image.
Optionally, the binarization processing module includes:
carrying out graying processing on each image to be registered in the image set to be registered to obtain a plurality of grayscale images;
respectively calculating the gray level threshold value of each gray level image by a maximum inter-class difference method OTSU;
and aiming at each gray image, setting pixels with the gray values larger than the gray threshold value of the gray image in the gray image as first gray values, and setting pixels with the gray values not larger than the gray threshold value of the gray image in the gray image as second gray values, thereby obtaining a plurality of binary images.
Optionally, the connected domain detecting module includes:
the numerical value marking submodule is used for detecting the connected domain of each binary image and marking a numerical value for each pixel point in the binary image, wherein the numerical values of the pixel points in the same connected domain are equal;
the relabeling submodule is used for relabeling the numerical value of each pixel point in the binary image through a preset rule to obtain the relabeled numerical value of each pixel point;
and the connected domain division submodule is used for dividing the pixel points with the same numerical value into the same connected domain according to the re-marked numerical value of each pixel point to obtain one or more connected domains.
Optionally, the preset rule includes:
aiming at any pixel point, the numerical value of any pixel point is a first numerical value, and the numerical values of a plurality of pixel points adjacent to any pixel point are all second numerical values;
when the first numerical value is equal to the second numerical value, the numerical value of any pixel point is not marked again;
when the first numerical value is not equal to the second numerical value, the numerical value of any pixel point is marked as the second numerical value again.
Optionally, the preset rule includes:
aiming at any pixel point, the numerical value of any pixel point is a first numerical value, and when the numerical values of all the pixel points are not completely the same in a plurality of pixel points adjacent to any pixel point;
selecting a numerical value with the minimum numerical value of each pixel point from a plurality of adjacent pixel points of any pixel point as a target numerical value;
and marking the numerical value of any pixel point as a target numerical value.
Optionally, the target detection module includes:
the target identification submodule is used for identifying the target to be registered in each connected domain to obtain a binary image of the target to be registered in each binary image;
and the coordinate acquisition submodule is used for projecting the binary image of the target to be registered in each image to be registered to obtain the position coordinates of the target to be registered in each connected domain.
Optionally, the image fusion module includes:
the matrix acquisition submodule is used for calculating and obtaining a homography matrix between the images to be registered based on the coordinates of at least four points of the target to be registered;
the relation acquisition submodule is used for acquiring the mapping relation between the images to be registered according to the homography matrix between the images to be registered;
and the image fusion sub-module is used for registering and fusing the images to be registered in the image set to be registered according to the mapping relation among the images to be registered to obtain fused images.
The embodiment of the application also provides electronic equipment which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
a processor for implementing the image registration method of any of claims 1-7 when executing a program stored on the memory.
The present application further provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the image registration method according to any one of claims 1 to 7.
Embodiments of the present invention also provide a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the image registration methods described above.
The embodiment of the invention has the following beneficial effects:
the image registration method, the image registration device, the electronic equipment and the storage medium provided by the embodiment of the invention can acquire an image set to be registered; carrying out binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarization images; carrying out connected domain detection on each binary image to obtain one or more connected domains; detecting the target to be registered in each connected domain to obtain the position coordinates of the target to be registered in each connected domain; and registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered to obtain a fused image. Therefore, the coordinates of at least four points of the target to be registered of the multiple images to be registered are obtained, and the images to be registered in the image set to be registered are registered and fused according to the coordinates of the at least four points, so that the fused images with higher definition are obtained, and the details in the images are convenient to identify.
Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
Fig. 1 is a flowchart of an image registration method according to an embodiment of the present application;
fig. 2 is a flowchart illustrating a binarization process performed on each image to be registered according to an embodiment of the present application;
FIG. 3 is a flowchart illustrating connected component detection according to an embodiment of the present disclosure;
FIG. 4 is a flowchart illustrating detection of a target to be registered according to an embodiment of the present disclosure;
fig. 5 is a flowchart of registration and fusion of images to be registered according to the embodiment of the present application;
fig. 6 is a schematic diagram of an image registration apparatus according to an embodiment of the present application;
fig. 7 is an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
First, terms of art related to the present application are explained:
connected domain: that is, a Connected Component (Connected Component) generally refers to an image Region composed of foreground pixels having the same pixel value and adjacent positions in an image, and the corresponding english is Region or Blob.
Image registration: the process of matching or superposing two or more images acquired at different times, different devices or under different conditions (pose, climate, angle, etc.).
Projection histogram: the projection histogram is a graph obtained by plotting relative frequency of different levels of a variable by using rectangular blocks, and the projection histogram is mainly obtained by projecting a gradient image in the horizontal direction and the vertical direction in the patent of the invention.
In a first aspect of embodiments of the present application, there is provided an image registration method, including:
acquiring an image set to be registered, wherein the image set to be registered comprises at least two images to be registered, and each image to be registered comprises the same target to be registered;
carrying out binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarization images;
carrying out connected domain detection on each binary image to obtain one or more connected domains;
detecting the target to be registered in each connected domain to obtain the position coordinates of the target to be registered in each connected domain, wherein the position coordinates of the target to be registered comprise the coordinates of at least four points in the target to be registered;
and registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered to obtain a fused image.
Therefore, by the image registration method of the embodiment of the application, the coordinates of at least four points of the target to be registered of the plurality of images to be registered are obtained, and the images to be registered in the image set to be registered are registered and fused according to the coordinates of the at least four points, so that the fused images with higher definition are obtained, and the details in the images are convenient to identify.
In the following, a detailed description is given of an application scenario of the embodiment of the present application. When a plurality of pictures are acquired for a certain target, problems such as unclear certain details in the image may occur. In this case, the obtained multiple images may be registered and fused to obtain a fused image, and then the details in the image may be identified according to the fused image. For example:
when a traffic camera shoots a plurality of visible light or infrared light pictures of a certain vehicle and needs to identify details in images of a driver or a license plate and the like. Two visible light images or a visible light image and an infrared light image can be selected, then the selected images are registered and fused to obtain an image with higher definition, and then details in the images of a driver or a license plate and the like are identified according to the fused image.
Referring to fig. 1, fig. 1 is a flowchart of an image registration method according to an embodiment of the present application, including:
and step S11, acquiring an image set to be registered.
The image set to be registered comprises at least two images to be registered, each image to be registered comprises the same target to be registered, and the images to be registered can be visible light images, infrared light images and the like. For example, when a traffic camera is used for capturing a certain automobile driver, at least two pictures for the front window of the automobile are captured continuously, and in the pictures for the front window of the automobile, the target to be registered can be the front window of the automobile. For another example, when a traffic camera captures front and rear license plates of a certain automobile, at least two pictures of the front and rear license plates of the automobile are captured continuously, and the target to be registered is the front and rear license plates of the automobile.
The embodiment of the application aims at the image to be registered in the intelligent terminal, so that the image can be executed through the intelligent terminal, and particularly, the intelligent terminal can be an intelligent camera, a hard disk video recorder, a computer or a server and the like.
And step S12, performing binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarized images.
The binarization processing is performed on the image to be registered, and may be to perform graying on the image to be registered first, set each pixel point in the image to be registered to a certain numerical value between 0 and 255, then divide the image into a background and a foreground by assumption, calculate to obtain a grayscale threshold of the image to be registered, and re-mark the numerical value of each pixel point in the image to be registered through the grayscale threshold. For example, pixels less than the gray threshold may be labeled as 0, and pixels greater than the gray threshold may be labeled as 255. The specific binarization processing method may refer to a binarization method in the related art, for example, OTSU (maximum inter-class difference) binarization processing.
And step S13, performing connected domain detection on each binary image to obtain one or more connected domains.
And respectively detecting the connected domain of each binary image. For example, a pixel point larger than the gray threshold in the binarized image may be marked as 1, and a pixel point smaller than the gray threshold may be marked as 0 by a two-pass scanning method, so as to mark a numerical value for each pixel point in the binarized image, and perform connected domain detection according to the adjacent relationship between the marked numerical value and the pixel point, so as to obtain one or more connected domains in the binarized image.
And step S14, detecting the target to be registered for each connected domain to obtain the position coordinates of the target to be registered for each connected domain.
The position coordinates of the target to be registered comprise coordinates of at least four points in the target to be registered. The detection of the target to be registered is performed on each connected domain, and may be performed on each connected domain according to the shape of the target to be registered, so as to obtain an image of the target to be registered in one or more connected domains, and further obtain position coordinates of at least four points of the image of the target to be registered in the image to be registered.
For example, when a plurality of pictures of a certain automobile front window shot by a traffic camera are registered, the target to be registered is the automobile front window, and the coordinates of the target to be registered may be the coordinates of four vertexes of the automobile front window. For another example, when images of a plurality of front and rear license plates of an automobile shot by a traffic camera are registered, the target to be registered is the front and rear license plates of the automobile, and the coordinates of the target to be registered can be coordinates of four vertexes of the front and rear license plates of the automobile.
And step S15, registering and fusing the images to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered to obtain fused images.
The registration and fusion of the images to be registered in the image set to be registered are performed based on the coordinates of at least four points of the target to be registered, and the registration of the images to be registered in the image set to be registered can be performed according to the corresponding relationship of the coordinates of at least four points of the images to be registered in the plurality of images to be registered. For example, if the coordinates of the four points of the target to be registered include coordinates of four upper, lower, left, and right points of the target to be registered, the four upper, lower, left, and right points of one image to be registered are taken as a reference, the four upper, lower, left, and right points of other images to be registered are respectively registered with the four upper, lower, left, and right points of the reference image to be registered, and the registered images are fused to obtain a fused image.
For example, according to coordinates of four vertexes of the automobile front window, registration and fusion are performed on a plurality of images of the automobile front window, and an automobile front window image with higher definition after fusion is obtained. For another example, the front and rear license plates of the automobile are fused according to coordinates of four vertexes of the front and rear license plates of the automobile, so that the fused images of the front and rear license plates of the automobile with higher definition are obtained.
Therefore, by the image registration method of the embodiment of the application, the coordinates of at least four points of the target to be registered of the plurality of images to be registered are obtained, and the images to be registered in the image set to be registered are registered and fused according to the coordinates of the at least four points, so that the fused images with higher definition are obtained, and the details in the images are convenient to identify.
Optionally, referring to fig. 2, in step S12, performing binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarized images, where the binarization processing includes:
and S121, performing graying processing on each image to be registered in the image set to be registered to obtain a plurality of grayscale images.
The graying of the image to be registered can be achieved in various ways, for example, graying of each image to be registered in the image set to be registered can be achieved through Gamma correction, and a grayscale image corresponding to each image to be registered is obtained, so that a plurality of grayscale images are obtained. The graying process may set each pixel in the image to be registered to a value between 0 and 255.
And step S122, respectively calculating the gray threshold of each gray image through an OTSU algorithm.
The OTSU algorithm assumes that there is a threshold Th to classify all pixels of the image into two classes, C1 (smaller than Th) and C2 (larger than Th), and the respective mean values of these two classes of pixels are m1 and m2, and the global mean value of the image is mG. The probability of simultaneous pixels being classified into classes C1 and C2 is p1, p2, respectively. Thus, there are:
p1*m1+p2*m2=mG
p1+p2=1
the expression of class variance is:
σ2=p1(m1-mG)2+p2(m2-mG)2
thereby obtaining:
σ2=p1*p2(m1-m2)2
when the variance of the above formula is maximum, the obtained mean value mG is the threshold value TH, and in the formula:
Figure BDA0002673743500000101
Figure BDA0002673743500000102
Figure BDA0002673743500000103
wherein, L is the total gray scale number, and i is the number of pixel points.
Step S123, for each gray image, setting the pixels in the gray image with the gray values greater than the gray threshold of the gray image as the first gray values, and setting the pixels in the gray image with the gray values not greater than the gray threshold of the gray image as the second gray values, thereby obtaining a plurality of binary images.
Pixels in the grayscale image having a grayscale value not greater than the grayscale image grayscale threshold are set to a second grayscale value, e.g., 0, by setting pixels in the grayscale image having a grayscale value greater than the grayscale image grayscale threshold to a first grayscale value, e.g., 255. Thereby facilitating the inspection of the connected domain of the binary image.
Optionally, referring to fig. 3, step S13 performs connected domain detection on each binarized image to obtain one or more connected domains, including:
step S131, connected domain detection is carried out on each binary image, and a numerical value is marked for each pixel point in the binary image.
For example, scanning the binary image, and setting a label value for each effective pixel, for example, marking the pixel greater than the gray threshold in the binary image as 1, and marking the pixel less than the gray threshold as 0.
Or by the following rules:
1) aiming at a certain pixel point, when the cable of the left adjacent pixel point and the upper adjacent pixel point of the pixel is an invalid value, namely the cable is not set, the pixel point is set to be a new label value, namely label + +;
2) aiming at a certain pixel point, when a left adjacent pixel point or an upper adjacent pixel point of the pixel point has an effective value, setting the label value of the pixel to be the same as the label of the pixel point with the effective value;
3) and aiming at a certain pixel point, when the left adjacent pixel point and the upper adjacent pixel point of the pixel point are effective values, setting the label value of the pixel to be the same as the smaller label value.
Step S132, the numerical value of each pixel point in the binary image is re-marked through a preset rule, and the re-marked numerical value of each pixel point is obtained.
In the actual use process, the numerical value of each pixel point in the binary image is re-marked through a preset rule, so that labels belonging to the same connected region but having different values can be marked as the same leble, that is, the equality relationship between the labels and the lebel is recorded.
Step S133, according to the relabeled value of each pixel, dividing the pixels with the same value into the same connected domain to obtain one or more connected domains.
In the actual use process, the pixels with the same value are divided into the same connected domain according to the re-marked value of each pixel, for example, the recorded pixels with the equal relationship are classified into one connected domain and are given the same label.
Therefore, the connected domain detection method provided by the embodiment of the application can be used for not only detecting the connected domains of the binary images, but also repartitioning the connected domains of the pixel points belonging to the same connected domain to obtain one or more connected domains.
Optionally, the preset rule includes: aiming at any pixel point, the numerical value of any pixel point is a first numerical value, and the numerical values of a plurality of pixel points adjacent to any pixel point are all second numerical values; when the first numerical value is equal to the second numerical value, the numerical value of any pixel point is not marked again; when the first numerical value is not equal to the second numerical value, the numerical value of any pixel point is marked as the second numerical value again.
Optionally, the preset rule includes: aiming at any pixel point, the numerical value of any pixel point is a first numerical value, and when the numerical values of all the pixel points are not completely the same in a plurality of pixel points adjacent to any pixel point; selecting a numerical value with the minimum numerical value of each pixel point from a plurality of adjacent pixel points of any pixel point as a target numerical value; and marking the numerical value of any pixel point as a target numerical value.
For example, the image is scanned by the method described above:
(1) first scanning:
accessing the current pixel B (x, y), if B (x, y) is 1:
if the pixel values in the domain of B (x, y) are all 0, then B (x, y) is given a new label:
label+=1,B(x,y)=label;
if there are pixels Neighbors with pixel values >1 in the field of B (x, y):
1) assigning the minimum in Neighbors to B (x, y):
B(x,y)=min{Neighbors}
2) recording the equality relation among all values (label) in the neighbor, namely the values (label) belong to the same connected region;
labelSet [ i ] ═ label _ m., label _ n }, and all labels in labelSet [ i ] belong to the same connected region (note: there can be various implementations as long as the relationship between those labels with equal relationship can be recorded)
(2) And (3) second scanning:
access the current pixel B (x, y), if B (x, y) > 1:
finding a minimum label value which is in the same generic relation with label ═ B (x, y), and assigning B (x, y);
after the scanning is completed, the pixels with the same label value in the image form the same connected region.
Therefore, through the preset rule of the embodiment of the application, the connected domain to which the pixel belongs can be judged, and the pixel belonging to the same connected domain is subjected to connected domain repartitioning to obtain one or more connected domains.
Optionally, referring to fig. 4, the step S14 of detecting the target to be registered for each connected domain to obtain the position coordinates of the target to be registered for each connected domain includes:
step S141, identifying the target to be registered for each connected domain, and obtaining a binary image of the target to be registered in each binary image.
The target to be registered is identified for each connected domain, and the identification can be performed according to the shape and the like of the target to be registered, so that a binary image of the target to be registered in each binary image is obtained. For example, when the object to be registered is a front window of an automobile, connected domains with a trapezoidal shape in each connected domain can be identified to obtain a binary image corresponding to the front window of the automobile, and for example, when the object to be registered is a front license plate and a rear license plate of the automobile, connected domains with a rectangular shape in each connected domain can be identified to obtain a binary image corresponding to the license plate of the automobile.
Step S142, projecting the binary image of the target to be registered in each image to be registered to obtain the position coordinates of the target to be registered in each connected domain.
The binary image of the target to be registered in each image to be registered is projected, a horizontal histogram projection can be performed on the binary image of the target to be registered to obtain a coordinate in the vertical direction, and then the binary image is counted to obtain a horizontal coordinate.
For example, the position coordinates of the target to be registered include coordinates of four points, and the target to be registered is a window of an automobile. And after obtaining the horizontal projection histogram, counting conversion values of w/2 columns of pixels, wherein positions where two jumps occur are upper and lower coordinates y1 and y2 of the window, counting the change rule of pixel values at (y1+ y2)/2 rows in a connected domain binary image, and obtaining the x coordinates, x1 and x2 of the left and right outlines of the window according to the change of the pixel values from white to black. Four vertex coordinates P1(x1, y1), P2(x1, y2), P3(x2, y1) and P4(x4, y4) of the plane where the vehicle window is located are obtained according to the four coordinates.
Therefore, by the method for detecting the target to be registered in each connected domain, the target to be registered can be identified and projected in each connected domain, and the position coordinates of the target to be registered in each connected domain can be obtained, so that the images to be registered can be conveniently registered and fused according to the position coordinates of the target to be registered, and the fused images can be obtained.
Optionally, referring to fig. 5, in step S15, based on the coordinates of at least four points of the target to be registered, the images to be registered in the image set to be registered are registered and fused, so as to obtain a fused image, where the method includes:
step S151, calculating to obtain a homography matrix between the images to be registered based on the coordinates of at least four points of the target to be registered.
For example, the object to be registered is an automobile window, the image set to be registered comprises two photos of the same automobile, 4 vertex coordinates P1-P4 of the window area are obtained, and then the coordinate of one point in four points in the previous frame is (u)f,vf) And the coordinate of one point of the four points in the next frame is (u)s,vs) Calculating to obtain a homography matrix H mapped to a next frame;
Figure BDA0002673743500000141
step S152, according to the homography matrix among the images to be registered, the mapping relation among the images to be registered is obtained.
According to the above example, the target to be registered is an automobile window, and the two images are mapped according to the homography matrix H between the images to be registered
Figure BDA0002673743500000142
Is provided with
Figure BDA0002673743500000143
Figure BDA0002673743500000144
Step S153, registering and fusing the images to be registered in the image set to be registered according to the mapping relation among the images to be registered to obtain fused images.
It can be seen that, by the coordinate of at least four points of the target to be registered according to the embodiment of the present application, the homography matrix between the images to be registered is obtained through calculation, so as to obtain the mapping relationship between the images to be registered, and according to the mapping relationship between the images to be registered, the images to be registered in the image set to be registered are registered and fused, so that the registration and fusion of a plurality of images to be registered can be performed, and therefore, an image with higher definition compared with the images to be registered can be obtained, and thus, the identification of details in the images to be registered can be performed.
Referring to fig. 6, in a second aspect of the embodiments of the present application, there is provided an image registration apparatus including:
the image acquiring module 601 is configured to acquire an image set to be registered, where the image set to be registered includes at least two images to be registered, and each image to be registered includes a same target to be registered;
a binarization processing module 602, configured to perform binarization processing on each to-be-registered image in the to-be-registered image set to obtain multiple binarization images;
a connected domain detection module 603, configured to perform connected domain detection on each binarized image to obtain one or more connected domains;
the target detection module 604 is configured to detect the target to be registered in each connected domain, and obtain position coordinates of the target to be registered in each connected domain, where the position coordinates of the target to be registered include coordinates of at least four points in the target to be registered;
the image fusion module 605 is configured to perform registration and fusion on each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered, so as to obtain a fused image.
Optionally, the binarization processing module 602 includes:
carrying out graying processing on each image to be registered in the image set to be registered to obtain a plurality of grayscale images;
respectively calculating the gray level threshold value of each gray level image by a maximum inter-class difference method OTSU;
and aiming at each gray image, setting pixels with the gray values larger than the gray threshold value of the gray image in the gray image as first gray values, and setting pixels with the gray values not larger than the gray threshold value of the gray image in the gray image as second gray values, thereby obtaining a plurality of binary images.
Optionally, the connected component detection module 603 includes:
the numerical value marking submodule is used for detecting the connected domain of each binary image and marking a numerical value for each pixel point in the binary image, wherein the numerical values of the pixel points in the same connected domain are equal;
the relabeling submodule is used for relabeling the numerical value of each pixel point in the binary image through a preset rule to obtain the relabeled numerical value of each pixel point;
and the connected domain division submodule is used for dividing the pixel points with the same numerical value into the same connected domain according to the re-marked numerical value of each pixel point to obtain one or more connected domains.
Optionally, the preset rule includes:
aiming at any pixel point, the numerical value of any pixel point is a first numerical value, and the numerical values of a plurality of pixel points adjacent to any pixel point are all second numerical values;
when the first numerical value is equal to the second numerical value, the numerical value of any pixel point is not marked again;
when the first numerical value is not equal to the second numerical value, the numerical value of any pixel point is marked as the second numerical value again.
Optionally, the preset rule includes:
aiming at any pixel point, the numerical value of any pixel point is a first numerical value, and when the numerical values of all the pixel points are not completely the same in a plurality of pixel points adjacent to any pixel point;
selecting a numerical value with the minimum numerical value of each pixel point from a plurality of adjacent pixel points of any pixel point as a target numerical value;
and marking the numerical value of any pixel point as a target numerical value.
Optionally, the target detection module 604 includes:
the target identification submodule is used for identifying the target to be registered in each connected domain to obtain a binary image of the target to be registered in each binary image;
and the coordinate acquisition submodule is used for projecting the binary image of the target to be registered in each image to be registered to obtain the position coordinates of the target to be registered in each connected domain.
Optionally, the image fusion module 605 includes:
the matrix acquisition submodule is used for calculating and obtaining a homography matrix between the images to be registered based on the coordinates of at least four points of the target to be registered;
the relation acquisition submodule is used for acquiring the mapping relation between the images to be registered according to the homography matrix between the images to be registered;
and the image fusion sub-module is used for registering and fusing the images to be registered in the image set to be registered according to the mapping relation among the images to be registered to obtain fused images.
Therefore, by the image registration device of the embodiment of the application, the coordinates of at least four points of the target to be registered of the plurality of images to be registered are obtained, and the images to be registered in the image set to be registered are registered and fused according to the coordinates of the at least four points, so that the fused images with higher definition are obtained, and the details in the images are convenient to identify.
An embodiment of the present invention further provides an electronic device, as shown in fig. 7, including a processor 701, a communication interface 702, a memory 703 and a communication bus 704, where the processor 701, the communication interface 702, and the memory 703 complete mutual communication through the communication bus 704,
a memory 703 for storing a computer program;
the processor 701 is configured to implement any of the image registration methods described above when executing the program stored in the memory 703.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In a further embodiment provided by the present invention, there is also provided a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of any of the image registration methods described above.
In a further embodiment provided by the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the image registration methods of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the device, since it is basically similar to the method embodiment, the description is simple, and the relevant points can be referred to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (10)

1. An image registration method, comprising:
acquiring an image set to be registered, wherein the image set to be registered comprises at least two images to be registered, and each image to be registered comprises the same target to be registered;
carrying out binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarization images;
performing connected domain detection on each binary image to obtain one or more connected domains;
detecting the target to be registered of each connected domain to obtain the position coordinates of the target to be registered of each connected domain, wherein the position coordinates of the target to be registered comprise the coordinates of at least four points in the target to be registered;
and registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered to obtain a fused image.
2. The method according to claim 1, wherein the binarizing processing is performed on each image to be registered in the image set to be registered to obtain a plurality of binarized images, and comprises:
carrying out graying processing on each image to be registered in the image set to be registered to obtain a plurality of grayscale images;
respectively calculating the gray level threshold value of each gray level image by a maximum inter-class difference method OTSU;
and aiming at each gray image, setting pixels with the gray values larger than the gray threshold value of the gray image in the gray image as first gray values, and setting pixels with the gray values not larger than the gray threshold value of the gray image in the gray image as second gray values, thereby obtaining a plurality of binary images.
3. The method according to claim 1, wherein the performing connected component detection on each of the binarized images to obtain one or more connected components comprises:
detecting connected domains of the binary images, and marking a numerical value for each pixel point in the binary images, wherein the numerical values of the pixel points in the same connected domain are equal;
re-marking the numerical value of each pixel point in the binary image according to a preset rule to obtain a re-marked numerical value of each pixel point;
and dividing the pixels with the same value into the same connected domain according to the re-marked value of each pixel to obtain one or more connected domains.
4. The method of claim 3, wherein the preset rules comprise:
aiming at any pixel point, the numerical value of the pixel point is a first numerical value, and the numerical values of a plurality of pixel points adjacent to the pixel point are all second numerical values;
when the first numerical value is equal to the second numerical value, the numerical value of any pixel point is not marked again;
when the first numerical value is not equal to the second numerical value, the numerical value of any pixel point is marked as the second numerical value again.
5. The method according to claim 3 or 4, wherein the preset rule comprises:
aiming at any pixel point, the numerical value of the pixel point is a first numerical value, and when the numerical values of the pixel points are not completely the same in a plurality of pixel points adjacent to the pixel point;
selecting a numerical value with the minimum numerical value of each pixel point from the plurality of adjacent pixel points as a target numerical value;
and marking the value of any pixel point as the target value.
6. The method of claim 1, wherein the detecting the target to be aligned for each of the connected domains to obtain the position coordinates of the target to be aligned for each of the connected domains comprises:
identifying the target to be registered for each connected domain to obtain a binary image of the target to be registered in each binary image;
and projecting the binary image of the target to be registered in each image to be registered to obtain the position coordinates of the target to be registered in each connected domain.
7. The method according to claim 1, wherein the registering and fusing each image to be registered in the image set to be registered based on coordinates of at least four points of the target to be registered to obtain a fused image comprises:
calculating to obtain a homography matrix between the images to be registered based on the coordinates of at least four points of the target to be registered;
obtaining a mapping relation between the images to be registered according to the homography matrix between the images to be registered;
and registering and fusing the images to be registered in the image set to be registered according to the mapping relation among the images to be registered to obtain fused images.
8. An image registration apparatus, comprising:
the image registration system comprises an image acquisition module, a registration module and a registration module, wherein the image acquisition module is used for acquiring an image set to be registered, the image set to be registered comprises at least two images to be registered, and each image to be registered comprises the same target to be registered;
a binarization processing module, configured to perform binarization processing on each to-be-registered image in the to-be-registered image set to obtain multiple binarization images;
the connected domain detection module is used for detecting the connected domains of the binary images to obtain one or more connected domains;
the target detection module is used for detecting the target to be registered in each connected domain to obtain the position coordinates of the target to be registered in each connected domain, wherein the position coordinates of the target to be registered comprise the coordinates of at least four points in the target to be registered;
and the image fusion module is used for registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered to obtain a fused image.
9. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 1 to 7 when executing a program stored in the memory.
10. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 7.
CN202010941346.7A 2020-09-09 2020-09-09 Image registration method and device, electronic equipment and storage medium Active CN112017218B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010941346.7A CN112017218B (en) 2020-09-09 2020-09-09 Image registration method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010941346.7A CN112017218B (en) 2020-09-09 2020-09-09 Image registration method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN112017218A true CN112017218A (en) 2020-12-01
CN112017218B CN112017218B (en) 2024-08-02

Family

ID=73522131

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010941346.7A Active CN112017218B (en) 2020-09-09 2020-09-09 Image registration method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112017218B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112668676A (en) * 2020-12-12 2021-04-16 浙江大华技术股份有限公司 Package security inspection image processing method, electronic device and storage medium

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106097344A (en) * 2016-06-15 2016-11-09 武汉理工大学 A kind of image processing method detecting geometric form impurity in rubber for tire and system
CN107833238A (en) * 2017-11-14 2018-03-23 京东方科技集团股份有限公司 Largest connected field mark method, method for tracking target, augmented reality/virtual reality device
CN108345888A (en) * 2018-02-11 2018-07-31 浙江华睿科技有限公司 A kind of connected domain extracting method and device
CN108510455A (en) * 2018-03-27 2018-09-07 长春理工大学 A kind of laser irradiation device image interfusion method and system
CN109166077A (en) * 2018-08-17 2019-01-08 广州视源电子科技股份有限公司 Image alignment method and device, readable storage medium and computer equipment
CN109191415A (en) * 2018-08-22 2019-01-11 成都纵横自动化技术股份有限公司 Image interfusion method, device and electronic equipment
CN109509200A (en) * 2018-12-26 2019-03-22 深圳市繁维医疗科技有限公司 Checkerboard angle point detection process, device and computer readable storage medium based on contours extract
CN110189368A (en) * 2019-05-31 2019-08-30 努比亚技术有限公司 Method for registering images, mobile terminal and computer readable storage medium
CN110971841A (en) * 2019-12-09 2020-04-07 Oppo广东移动通信有限公司 Image processing method, image processing device, storage medium and electronic equipment
CN111028189A (en) * 2019-12-09 2020-04-17 Oppo广东移动通信有限公司 Image processing method, image processing device, storage medium and electronic equipment
CN111213178A (en) * 2019-03-29 2020-05-29 深圳市大疆创新科技有限公司 Connected domain processing method, data processing device and computer readable storage medium

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106097344A (en) * 2016-06-15 2016-11-09 武汉理工大学 A kind of image processing method detecting geometric form impurity in rubber for tire and system
CN107833238A (en) * 2017-11-14 2018-03-23 京东方科技集团股份有限公司 Largest connected field mark method, method for tracking target, augmented reality/virtual reality device
CN108345888A (en) * 2018-02-11 2018-07-31 浙江华睿科技有限公司 A kind of connected domain extracting method and device
CN108510455A (en) * 2018-03-27 2018-09-07 长春理工大学 A kind of laser irradiation device image interfusion method and system
CN109166077A (en) * 2018-08-17 2019-01-08 广州视源电子科技股份有限公司 Image alignment method and device, readable storage medium and computer equipment
CN109191415A (en) * 2018-08-22 2019-01-11 成都纵横自动化技术股份有限公司 Image interfusion method, device and electronic equipment
CN109509200A (en) * 2018-12-26 2019-03-22 深圳市繁维医疗科技有限公司 Checkerboard angle point detection process, device and computer readable storage medium based on contours extract
CN111213178A (en) * 2019-03-29 2020-05-29 深圳市大疆创新科技有限公司 Connected domain processing method, data processing device and computer readable storage medium
CN110189368A (en) * 2019-05-31 2019-08-30 努比亚技术有限公司 Method for registering images, mobile terminal and computer readable storage medium
CN110971841A (en) * 2019-12-09 2020-04-07 Oppo广东移动通信有限公司 Image processing method, image processing device, storage medium and electronic equipment
CN111028189A (en) * 2019-12-09 2020-04-17 Oppo广东移动通信有限公司 Image processing method, image processing device, storage medium and electronic equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
高梦珠: "基于图像配准的车标识别算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, pages 138 - 411 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112668676A (en) * 2020-12-12 2021-04-16 浙江大华技术股份有限公司 Package security inspection image processing method, electronic device and storage medium

Also Published As

Publication number Publication date
CN112017218B (en) 2024-08-02

Similar Documents

Publication Publication Date Title
US11455805B2 (en) Method and apparatus for detecting parking space usage condition, electronic device, and storage medium
Luvizon et al. A video-based system for vehicle speed measurement in urban roadways
Moghadam et al. Fast vanishing-point detection in unstructured environments
Pun et al. Multi-scale noise estimation for image splicing forgery detection
CN109978839B (en) Method for detecting wafer low-texture defects
TWI409718B (en) Method of locating license plate of moving vehicle
CN108564579B (en) Concrete crack detection method and detection device based on time-space correlation
CN105092598A (en) Method and system for rapidly recognizing defects of big-breadth PCB on basis of connected areas
WO2023185234A1 (en) Image processing method and apparatus, and electronic device and storage medium
CN109389122B (en) License plate positioning method and device
CN113505643B (en) Method and related device for detecting violation target
Paunwala et al. A novel multiple license plate extraction technique for complex background in Indian traffic conditions
CN112132892B (en) Target position labeling method, device and equipment
CN114155285B (en) Image registration method based on gray histogram
CN112017218B (en) Image registration method and device, electronic equipment and storage medium
CN114724119A (en) Lane line extraction method, lane line detection apparatus, and storage medium
US7440636B2 (en) Method and apparatus for image processing
CN114821274A (en) Method and device for identifying state of split and combined indicator
CN110765875B (en) Method, equipment and device for detecting boundary of traffic target
Lin et al. Vision-based patient identification recognition based on image content analysis and support vector machine for medical information system
CN112052859A (en) License plate accurate positioning method and device in free scene
CN115083008A (en) Moving object detection method, device, equipment and storage medium
Powell et al. Automated road distress detection
Setiyorini et al. Vehicle number plate identification using template matching algorithm for automatic parking system
CN111695374A (en) Method, system, medium, and apparatus for segmenting zebra crossing region in monitoring view

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant