CN109584197A - A kind of image interfusion method and relevant apparatus - Google Patents
A kind of image interfusion method and relevant apparatus Download PDFInfo
- Publication number
- CN109584197A CN109584197A CN201811563609.4A CN201811563609A CN109584197A CN 109584197 A CN109584197 A CN 109584197A CN 201811563609 A CN201811563609 A CN 201811563609A CN 109584197 A CN109584197 A CN 109584197A
- Authority
- CN
- China
- Prior art keywords
- image
- registration
- image data
- fused
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000004927 fusion Effects 0.000 claims abstract description 68
- 241001269238 Data Species 0.000 claims abstract description 30
- 238000002156 mixing Methods 0.000 claims abstract description 14
- 230000009466 transformation Effects 0.000 claims description 40
- 238000004422 calculation algorithm Methods 0.000 claims description 24
- 238000003708 edge detection Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 10
- 239000000155 melt Substances 0.000 claims description 4
- 241000208340 Araliaceae Species 0.000 claims description 3
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 claims description 3
- 235000003140 Panax quinquefolius Nutrition 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 3
- 235000008434 ginseng Nutrition 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 21
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 26
- 238000011282 treatment Methods 0.000 description 10
- 238000000354 decomposition reaction Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000005267 amalgamation Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20064—Wavelet transform [DWT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
This application discloses a kind of image interfusion methods, comprising: FPGA receives multiple image datas to be fused that server is sent；Image registration is carried out to the multiple image data to be fused, obtains the registration parameter of each image data to be fused；Image co-registration is carried out to the multiple image data to be fused according to all registration parameters, obtains fusion image data；The fusion image data is sent to the server.Image co-registration is realized by FPGA, the characteristics of FPGA Programmadle logic array is utilized, image co-registration process is accelerated, improve the efficiency of image co-registration, server is allowed faster to get blending image, and the hardware performance of server itself is saved, server hardware utilization rate must as far as possible maximized.Disclosed herein as well is a kind of image fusion device, image co-registration equipment and computer readable storage mediums, have above-mentioned beneficial effect.
Description
Technical field
This application involves technical field of image processing, in particular to a kind of image interfusion method, image fusion device, image
Fusion device and computer readable storage medium.
Background technique
With the continuous development of information technology, development of Mobile Internet technology deepens continuously the life of consumer, so that internet
In image data it is more and more.The demand for carrying out fusion treatment for image data is also just more and more.Wherein, image co-registration
Refer to by multi-source channel the collected image data about same target generated after fusion treatment comprising more information
New images.The utilization rate of image information can be improved, improve computer interpretation precision and reliability, promote the sky of original image
Between the characteristics such as resolution ratio and spectral resolution.
Currently, being directed to a small amount of image data in the prior art, image co-registration processing is generally carried out by the way of software,
Namely image co-registration processing is carried out using CPU (Central Processing Unit central processing unit) on computers.But
It is when the image data for needing to carry out image co-registration processing becomes more, image co-registration processing is carried out by way of software has expired
The foot not rate request of fusion treatment, the efficiency of image co-registration processing is extremely low, and seriously occupies the hardware resource of server,
Reduce performance utilization rate.
Therefore, how to improve the speed of image co-registration processing is the Important Problems of those skilled in the art's concern.
Summary of the invention
The purpose of the application is to provide a kind of image interfusion method, image fusion device, image co-registration equipment and calculating
Machine readable storage medium storing program for executing, the characteristics of realizing image co-registration by FPGA, FPGA Programmadle logic array is utilized, to image co-registration
Process is accelerated, and the efficiency of image co-registration is improved.
In order to solve the above technical problems, the application provides a kind of image interfusion method, comprising:
FPGA receives multiple image datas to be fused that server is sent；
Image registration is carried out to the multiple image data to be fused, obtains the registration of each image data to be fused
Parameter；
Image co-registration is carried out to the multiple image data to be fused according to all registration parameters, obtains fusion figure
As data；
The fusion image data is sent to the server.
Optionally, image registration is carried out to the multiple image data to be fused, obtains each picture number to be fused
According to registration parameter, comprising:
The maximum transformation parameter of mutual information between each image data to be fused is searched using mutual information algorithm, it will be every
A image data to be fused corresponding maximum transformation parameter of mutual information matching as each image data to be fused
Quasi- parameter.
Optionally, image registration is carried out to the multiple image data to be fused, obtains each picture number to be fused
According to registration parameter, comprising:
Edge detection is carried out to each image data to be fused according to Sobel operator, obtains multiple marginal informations；
The maximum transformation parameter of mutual information between each marginal information is searched using mutual information algorithm, it will be each described
Registration parameter of the maximum transformation parameter of the corresponding mutual information of marginal information as each image data to be fused.
Optionally, image co-registration is carried out to the multiple image data to be fused according to all registration parameters, obtained
Fusion image data, comprising:
Image transformation is carried out to corresponding image data to be fused according to each registration parameter, obtains multiple registration figures
As data；
Image co-registration is carried out to the multiple registration image data according to Algorithms of Discrete Wavelet Transform, obtains described merged
Image data.
The application also provides a kind of image fusion device, comprising:
Image data receiving module, for receiving multiple image datas to be fused of server transmission；
Image registration module, for carrying out image registration to the multiple image data to be fused, obtain it is each it is described to
The registration parameter of fusion image data；
Image co-registration module, for carrying out image to the multiple image data to be fused according to all registration parameters
Fusion, obtains fusion image data；
Blending image sending module, for the fusion image data to be sent to the server.
Optionally, described image registration module is specifically used for searching each image to be fused using mutual information algorithm
The maximum transformation parameter of mutual information between data joins the corresponding maximum transformation of mutual information of each image data to be fused
Registration parameter of the number as each image data to be fused.
Optionally, described image registration module, comprising:
Edge detection unit is obtained for carrying out edge detection to each image data to be fused according to Sobel operator
To multiple marginal informations；
Mutual information registration unit, it is maximum for searching mutual information between each marginal information using mutual information algorithm
Transformation parameter, using the corresponding maximum transformation parameter of mutual information of each marginal information as each picture number to be fused
According to registration parameter.
Optionally, described image Fusion Module, comprising:
Image transforming unit, for carrying out image change to corresponding image data to be fused according to each registration parameter
It changes, obtains multiple registration image datas；
Wavelet Transform Fusion unit, for carrying out figure to the multiple registration image data according to Algorithms of Discrete Wavelet Transform
As fusion, the fusion image data is obtained.
The application also provides a kind of image co-registration equipment, comprising:
Memory, for storing computer program；
Processor, the step of image interfusion method as described above is realized when for executing the computer program.
The application also provides a kind of computer readable storage medium, and calculating is stored on the computer readable storage medium
The step of machine program, the computer program realizes image interfusion method as described above when being executed by processor.
A kind of image interfusion method provided herein, comprising: FPGA receives multiple figures to be fused that server is sent
As data；Image registration is carried out to the multiple image data to be fused, obtains the registration of each image data to be fused
Parameter；Image co-registration is carried out to the multiple image data to be fused according to all registration parameters, obtains blending image
Data；The fusion image data is sent to the server.
Image registrations are carried out to multiple image datas to be fused by FPGA, according to obtained registration parameter to needing to be melted
It closes image data and carries out image co-registration, obtain fusion image data, then fusion image data is back in server,
It is exactly the characteristics of server is realized image co-registration, FPGA Programmadle logic array is utilized using FPGA, to image co-registration mistake
Cheng Jinhang accelerates, and improves the efficiency of image co-registration, allows server faster to get blending image, and save
The about hardware performance of server itself as far as possible must be such that server hardware utilization rate maximizes.
The application also provides a kind of image fusion device, image co-registration equipment and computer readable storage medium, has
Above-mentioned beneficial effect, this will not be repeated here.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of application for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow chart of image interfusion method provided by the embodiment of the present application；
Fig. 2 is a kind of structural schematic diagram of image fusion device provided by the embodiment of the present application.
Specific embodiment
The core of the application is to provide a kind of image interfusion method, image fusion device, image co-registration equipment and calculating
Machine readable storage medium storing program for executing, the characteristics of realizing image co-registration by FPGA, FPGA Programmadle logic array is utilized, to image co-registration
Process is accelerated, and the efficiency of image co-registration is improved.
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application
In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is
Some embodiments of the present application, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art
Every other embodiment obtained without making creative work, shall fall in the protection scope of this application.
It is directed to a small amount of image data in the prior art, generally carries out image co-registration processing by the way of software, also
It is to carry out image co-registration processing using CPU on computers.But when the image data for needing to carry out image co-registration processing becomes more
When, image co-registration processing is carried out by way of software can not meet the rate request of fusion treatment, image co-registration processing
Efficiency it is extremely low, and seriously occupy server hardware resource, reduce performance utilization rate.
Therefore, the application provides a kind of image interfusion method, carries out image to multiple image datas to be fused by FPGA
Registration carries out image co-registration to all image datas to be fused according to obtained registration parameter, obtains fusion image data, then
Fusion image data is back in server, that is, server realizes image co-registration using FPGA, and FPGA is utilized
The characteristics of Programmadle logic array, accelerates image co-registration process, improves the efficiency of image co-registration, so that server can
Faster to get blending image, and the hardware performance of server itself is saved, server must have been made hard as far as possible
Part utilization rate maximizes.
Referring to FIG. 1, Fig. 1 is a kind of flow chart of image interfusion method provided by the embodiment of the present application.
In the present embodiment, this method may include:
S101, FPGA (Field-Programmable Gate Array field programmable gate array) receive server hair
The multiple image datas to be fused sent；
This step is intended to that FPGA is made to get the multiple image datas to be fused for needing to carry out image co-registration from server.
It should be noted that mainly merging to two images in general image co-registration demand, that is, it incite somebody to action this
When embodiment is applied in more specifical environment, FPGA would only receive two images to be fused in single image fusion task
Data.But this does not illustrate that two images to be fused may only be carried out fusion treatments, therefore FPGA can receive to
The quantity of fusion image data simultaneously receives multiple image datas to be fused without limitation namely in this step.
S102 carries out image registration to multiple image datas to be fused, obtains the registration ginseng of each image data to be fused
Number；
On the basis of S101, this step is intended to multiple image datas to be fused carrying out image registration, so as to all
Image data to be fused realizes unified difference between each image to be fused of reduction under the same pixel coordinate system.Pass through
Obtained registration parameter carries out conversion process to each image data to be fused, so that it may the figure after obtaining the less registration of gap
Picture, to carry out corresponding image co-registration processing.
Wherein, registration parameter may include zooming parameter, rotation parameter and translation parameters.
The process being generally registrated is exactly to find to make the maximum transformation ginseng of similitude between two image datas to be fused
Number, when similitude maximum, indicates that the corresponding mapping mode of transformation parameter at this time can make between two images to be fused
Registration, the transformation parameter are registration parameter.
It should be noted that image registration can equally carry out between multiple image datas to be fused.Namely find
The maximum mapping mode of similitude between all image datas to be fused.
Further, the process that the maximum transformation parameter of similitude is found in this step, can use picture number to be fused
According to original image found.Equally, in order to improve the efficiency of searching process, that is, in order to improve the speed of image registration
Degree can be found using the corresponding marginal information of original image, can also using the corresponding bianry image of original image into
Row is found, and can also be found using the corresponding gray level image of original image, it is seen that the method for the image registration of this step is simultaneously
It is not unique, it can be selected, be not limited thereto according to the requirement of speed and accuracy to process of image registration.
S103 carries out image co-registration to multiple image datas to be fused according to all registration parameters, obtains blending image
Data；
On the basis of S102, this step aims at last image co-registration processing.Namely basis obtains all match
Quasi- parameter carries out image co-registration to multiple image datas to be fused.
Wherein, image co-registration processing can carry out image co-registration processing using Algorithms of Discrete Wavelet Transform, can also use
Any one image interfusion method that the prior art provides carries out image co-registration processing, therefore, not to specific in this step
The mode for carrying out image co-registration limits.
Fusion image data is sent to server by S104.
On the basis of S103, this step is intended to merge obtained fusion image data and is sent to server, so as to
Server carries out subsequent processing to the fusion image data received and either shows.
As it can be seen that by S101 to S103, so that the image co-registration operation carried out in the server originally, holds in FPGA
Row, the hardware feature that FPGA itself is utilized merge image data, improve the efficiency of image co-registration process.Also,
It should also be specified that carrying out image co-registration processing by FPGA to realize, need to get the corresponding programming language of FPGA
The program write, to be executed to corresponding program.Wherein, corresponding programming language includes but is not limited to HDL
(Open Computing Language is opened by (Hardware Description Language hardware description language) and OpenCL
Put operation language), wherein HDL can maximally utilize the hardware performance of FPGA, and OpenCL can be with Speeding up development speed.
Optionally, the S102 in the present embodiment may include:
The maximum transformation parameter of mutual information between each image data to be fused is searched using mutual information algorithm, will each to
Registration parameter of the maximum transformation parameter of the corresponding mutual information of fusion image data as each image data to be fused.
This optinal plan is mainly the mutual information being calculated by mutual information algorithm, is determined between image data to be fused
Similarity.It is maximum to find out similarity, that is, the maximum transformation parameter of mutual information, as each to be fused
The registration parameter of image data.
Optionally, the S102 in the present embodiment also may include:
Step 1, according to Sobel (the operator title of pixel image edge detection) operator to each image data to be fused into
Row edge detection obtains multiple marginal informations；
Step 2, the maximum transformation parameter of mutual information between each marginal information is searched using mutual information algorithm, by each side
Registration parameter of the maximum transformation parameter of the corresponding mutual information of edge information as each image data to be fused.
This optinal plan is mainly the marginal information by the corresponding original image of image data to be fused, is determined to be fused
Similarity between image data, and similarity is determined again by mutual information algorithm.
Optionally, the S103 in the present embodiment may include:
Step 1, image transformation is carried out to corresponding image data to be fused according to each registration parameter, obtains multiple registrations
Image data；
Step 2, image co-registration is carried out to multiple registration image datas according to Algorithms of Discrete Wavelet Transform, obtains fusion figure
As data.
This optinal plan mainly passes through Algorithms of Discrete Wavelet Transform and treats blending image progress image co-registration, to promote figure
As the effect of fusion.
To sum up, the present embodiment carries out image registration to multiple image datas to be fused by FPGA, according to obtained registration
Parameter carries out image co-registration to all image datas to be fused, obtains fusion image data, then fusion image data is returned
It is back in server, that is, server realizes image co-registration using FPGA, and the spy of FPGA Programmadle logic array is utilized
Point accelerates image co-registration process, improves the efficiency of image co-registration, and server is faster got
Blending image, and the hardware performance of server itself has been saved, server hardware utilization rate must as far as possible maximized.
On the basis of a upper embodiment, the present embodiment provides a kind of more specifical image interfusion methods.The present embodiment
The middle marginal information using original image carries out image registration, and the method for calculating similitude is improved using mutual information method
The speed and precision of image registration, and image co-registration processing is carried out using wavelet transform, improve the effect of image co-registration
Fruit.
The present embodiment proposes a kind of method for speeding up to realize rapid image fusion using FPGA, realizes that the hardware of this method is flat
Platform is FPGA accelerator card, the main treatment process that image co-registration is realized using HDL and OpenCL language.
Wherein, FPGA accelerator card is the PCIe board of half Gao Banchang of standard, can be inserted on the server, only takes up one
PCIe slot position.The board is developed based on the Arria10 Series FPGA of Intel, can support high-level programming language OpenCL, have
The advantages of high-performance low-power-consumption.
Server end will need the image merged to be transferred to FPGA accelerator card by PCIe interface, and FPGA completes fusion treatment
Fused image is returned to server end again afterwards, can be used for subsequent processing and display.
Image co-registration treatment process mainly includes that image registration and Pixel-level merge (namely image co-registration) two steps,
Wherein image registration belongs to image preprocessing process.Image registration is referred to matching the pixel coordinate of two images, be folded
The process for adding processing is to seek transition model or transformation relation from space aspects, allows image to be fused in the same pixel
Unification is realized under coordinate system, and then mitigates the difference among image.The Pixel-level fusion of image is referred to from not simultaneous interpretation
Sensor original image obtained gives the amalgamation mode directly integrated, it is believed that is to be included to each pixel of image
Information directly integrated.
Wherein, image registration is realized using image characteristics extraction, and specific method is using Sobel operator to two width
The edge feature of input picture extracts, and then calculates the registration parameter between two images by mutual information method, including
Zooming parameter, rotation parameter and translation parameters.Finally two images are carried out with the fusion of Pixel-level based on wavelet transformation.
Wherein, Sobel operator is a kind of difference operator of discrete type, for calculating the approximation of brightness of image functional gradient
Value.It includes two group 3 × 3 of matrix, respectively cross form Gx and vertical framework Gy, two templates is done with image respectively flat
Roll product, just can obtain horizontal and vertical brightness difference approximation.
Cross form:
Vertical framework:
The gradient magnitude of pixel gray level can be calculated using Gx and Gy are as follows:Pass through threshold again
Edge image can be obtained in value processing.Specific implementation method is: setting gray threshold T sets 0 for the pixel grey scale less than T,
Pixel grey scale greater than T is set as 1.
It is to carry out registration parameter calculating using mutual information method in next step after obtaining edge image.Two images are geometrically
What is be registrated is better, and a width figure includes that the information content of another width figure is bigger, i.e. the mutual information of two images is bigger.From original image
From the point of view of marginal information, two width edge registrations it is better, indicate two width original images mutual information it is bigger.As it can be seen that the process of registration
Be exactly find so that between two width edge images the maximum registration parameter of mutual information process.
The transformation model between two images is initially set up, the present invention selects rigid affine Transform Model.To be registered
It is as follows that image A and B establish rigid affine transformation relationship:
Wherein zooming parameter of the ρ between image A and B, θ are rotation parameter, ΔxAnd ΔyFor translation parameters.
The calculating process of mutual information is as follows:
The entropy of image A is defined as:
The entropy of image B is defined as:
The combination entropy of A and B is defined as:Wherein a ∈ A, b ∈ B；
The mutual information of image A and B may be defined as:
Wherein, PA(a)、PB(b) be respectively image A and B marginal probability distribution, PAB(a, b) is the joint of two images
Probability distribution.
By calculating image A and B in different ρ, θ, ΔxAnd ΔyUnder association relationship, when finding so that I (A, B) is maximum
Parameter value just realize the registration of two images as registration parameter.
After the registration for completing two images, the image co-registration of Pixel-level is carried out, the present invention is using based on discrete small
The fusion method of wave conversion, mainly comprises the steps of:
N layers of wavelet decomposition are carried out to image A and B respectively, establish the pyramid decomposition of two images；
Fusion treatment is carried out respectively to each decomposition layer, using different convergence strategies to the different fusion components of each decomposition layer
Fusion treatment is carried out, fused wavelet pyramid is obtained；
Wavelet inverse transformation is carried out to fused wavelet pyramid and carries out wavelet reconstruction, gained image is final after merging
Image.
During above-mentioned image co-registration, Sobel edge detection is realized using HDL language, image registration and is based on
The fusion of wavelet transformation is realized using OpenCL language.Program development, Neng Goujia are carried out using HDL language joint OpenCL
Fast development process, and more fully utilize the acceleration effect of FPGA.By actual test, the image that FPGA accelerator card is realized melts
Conjunction process can have 10 times or more of speed to be promoted relative to CPU processing.
The embodiment of the present application provides a kind of image interfusion method, can by FPGA to multiple image datas to be fused into
Row image registration carries out image co-registration to all image datas to be fused according to obtained registration parameter, obtains blending image
Data, then fusion image data is back in server, that is, server realizes image co-registration using FPGA, utilizes
The characteristics of FPGA Programmadle logic array, image co-registration process is accelerated, the efficiency of image co-registration is improved, so that clothes
Business device can faster get blending image, and save the hardware performance of server itself, must make to take as far as possible
Business device hardware utilization maximizes.
A kind of image fusion device provided by the embodiments of the present application is introduced below, a kind of image described below melts
It attaches together to set and can correspond to each other reference with a kind of above-described image interfusion method.
Referring to FIG. 2, Fig. 2 is a kind of structural schematic diagram of image fusion device provided by the embodiment of the present application.
In the present embodiment, the apparatus may include:
Image data receiving module 100, for receiving multiple image datas to be fused of server transmission；
Image registration module 200 obtains each figure to be fused for carrying out image registration to multiple image datas to be fused
As the registration parameter of data；
Image co-registration module 300, for carrying out image co-registration to multiple image datas to be fused according to all registration parameters,
Obtain fusion image data；
Blending image sending module 400, for fusion image data to be sent to server.
Optionally, the image registration module 200 specifically can be used for searching each image to be fused using mutual information algorithm
The maximum transformation parameter of mutual information between data makees the maximum transformation parameter of the corresponding mutual information of each image data to be fused
For the registration parameter of each image data to be fused.
Optionally, the image registration module 200 may include:
Edge detection unit obtains more for carrying out edge detection to each image data to be fused according to Sobel operator
A marginal information；
Mutual information registration unit, for searching the maximum transformation of mutual information between each marginal information using mutual information algorithm
Parameter is joined the maximum transformation parameter of the corresponding mutual information of each marginal information as the registration of each image data to be fused
Number.
Optionally, the image co-registration module 300 may include:
Image transforming unit, for carrying out image transformation to corresponding image data to be fused according to each registration parameter,
Obtain multiple registration image datas；
Wavelet Transform Fusion unit melts for carrying out image to multiple registration image datas according to Algorithms of Discrete Wavelet Transform
It closes, obtains fusion image data.
The embodiment of the present application also provides a kind of image co-registration equipment, comprising:
Memory, for storing computer program；
Processor realizes the step of image interfusion method as described above in Example when for executing the computer program
Suddenly.
The embodiment of the present application also provides a kind of computer readable storage medium, stores on the computer readable storage medium
There is computer program, the computer program realizes image interfusion method as described above in Example when being executed by processor
Step.
The computer readable storage medium may include: USB flash disk, mobile hard disk, read-only memory (Read-Only
Memory, ROM), random access memory (Random Access Memory, RAM), magnetic or disk etc. is various to deposit
Store up the medium of program code.
Each embodiment is described in a progressive manner in specification, the highlights of each of the examples are with other realities
The difference of example is applied, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment
Speech, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is referring to method part illustration
?.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond scope of the present application.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
Above to a kind of image interfusion method provided herein, image fusion device, image co-registration equipment and meter
Calculation machine readable storage medium storing program for executing is described in detail.Specific case used herein to the principle and embodiment of the application into
Elaboration is gone, the description of the example is only used to help understand the method for the present application and its core ideas.It should be pointed out that pair
For those skilled in the art, under the premise of not departing from the application principle, the application can also be carried out
Some improvements and modifications, these improvement and modification are also fallen into the protection scope of the claim of this application.
Claims (10)
1. a kind of image interfusion method characterized by comprising
FPGA receives multiple image datas to be fused that server is sent；
Image registration is carried out to the multiple image data to be fused, obtains the registration ginseng of each image data to be fused
Number；
Image co-registration is carried out to the multiple image data to be fused according to all registration parameters, obtains blending image number
According to；
The fusion image data is sent to the server.
2. image interfusion method according to claim 1, which is characterized in that carried out to the multiple image data to be fused
Image registration obtains the registration parameter of each image data to be fused, comprising:
The maximum transformation parameter of mutual information between each image data to be fused is searched using mutual information algorithm, by each institute
The registration that the maximum transformation parameter of the corresponding mutual information of image data to be fused is stated as each image data to be fused is joined
Number.
3. image interfusion method according to claim 1, which is characterized in that carried out to the multiple image data to be fused
Image registration obtains the registration parameter of each image data to be fused, comprising:
Edge detection is carried out to each image data to be fused according to Sobel operator, obtains multiple marginal informations；
The maximum transformation parameter of mutual information between each marginal information is searched using mutual information algorithm, by each edge
Registration parameter of the maximum transformation parameter of the corresponding mutual information of information as each image data to be fused.
4. image interfusion method according to any one of claims 1 to 3, which is characterized in that joined according to all registrations
It is several that image co-registration is carried out to the multiple image data to be fused, obtain fusion image data, comprising:
Image transformation is carried out to corresponding image data to be fused according to each registration parameter, obtains multiple registration picture numbers
According to；
Image co-registration is carried out to the multiple registration image data according to Algorithms of Discrete Wavelet Transform, obtains the blending image
Data.
5. a kind of image fusion device characterized by comprising
Image data receiving module, for receiving multiple image datas to be fused of server transmission；
Image registration module obtains each described to be fused for carrying out image registration to the multiple image data to be fused
The registration parameter of image data；
Image co-registration module is melted for carrying out image to the multiple image data to be fused according to all registration parameters
It closes, obtains fusion image data；
Blending image sending module, for the fusion image data to be sent to the server.
6. image fusion device according to claim 5, which is characterized in that described image registration module, specifically for adopting
The maximum transformation parameter of mutual information between each image data to be fused is searched with mutual information algorithm, it will be each described wait melt
Close registration parameter of the corresponding maximum transformation parameter of mutual information of image data as each image data to be fused.
7. image fusion device according to claim 5, which is characterized in that described image registration module, comprising:
Edge detection unit obtains more for carrying out edge detection to each image data to be fused according to Sobel operator
A marginal information；
Mutual information registration unit, for searching the maximum transformation of mutual information between each marginal information using mutual information algorithm
Parameter, using the corresponding maximum transformation parameter of mutual information of each marginal information as each image data to be fused
Registration parameter.
8. according to the described in any item image fusion devices of claim 5 to 7, which is characterized in that described image Fusion Module, packet
It includes:
Image transforming unit, for carrying out image transformation to corresponding image data to be fused according to each registration parameter,
Obtain multiple registration image datas；
Wavelet Transform Fusion unit melts for carrying out image to the multiple registration image data according to Algorithms of Discrete Wavelet Transform
It closes, obtains the fusion image data.
9. a kind of image co-registration equipment characterized by comprising
Memory, for storing computer program；
Processor realizes such as Claims 1-4 described in any item image interfusion methods when for executing the computer program
The step of.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program is realized when the computer program is executed by processor such as the described in any item image interfusion methods of Claims 1-4
Step.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811563609.4A CN109584197A (en) | 2018-12-20 | 2018-12-20 | A kind of image interfusion method and relevant apparatus |
| PCT/CN2019/103638 WO2020125062A1 (en) | 2018-12-20 | 2019-08-30 | Image fusion method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811563609.4A CN109584197A (en) | 2018-12-20 | 2018-12-20 | A kind of image interfusion method and relevant apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109584197A true CN109584197A (en) | 2019-04-05 |
Family
ID=65930197
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811563609.4A Pending CN109584197A (en) | 2018-12-20 | 2018-12-20 | A kind of image interfusion method and relevant apparatus |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109584197A (en) |
| WO (1) | WO2020125062A1 (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110633759A (en) * | 2019-09-24 | 2019-12-31 | 北京华力兴科技发展有限责任公司 | Image fusion method and device and electronic equipment |
| WO2020125062A1 (en) * | 2018-12-20 | 2020-06-25 | 广东浪潮大数据研究有限公司 | Image fusion method and related device |
| CN112272122A (en) * | 2020-10-14 | 2021-01-26 | 北京中科网威信息技术有限公司 | FPGA accelerator card detection method and device and readable storage medium |
| CN112819739A (en) * | 2021-01-28 | 2021-05-18 | 浙江祺跃科技有限公司 | Scanning electron microscope image processing method and system |
| CN115829897A (en) * | 2023-02-17 | 2023-03-21 | 湖北芯擎科技有限公司 | Image fusion processing method and device, electronic equipment and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104200460A (en) * | 2014-08-04 | 2014-12-10 | 西安电子科技大学 | Image registration method based on images characteristics and mutual information |
| CN107095384A (en) * | 2017-04-26 | 2017-08-29 | 长春理工大学 | A kind of Intelligent fire-fighting helmet device transmitted based on WIFI |
| CN206712926U (en) * | 2017-05-26 | 2017-12-05 | 淮阴师范学院 | A kind of more exposure image emerging systems |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9836433B1 (en) * | 2012-04-02 | 2017-12-05 | Rockwell Collins, Inc. | Image processing using multiprocessor discrete wavelet transform |
| CN108154494B (en) * | 2017-12-25 | 2019-05-14 | 北京航空航天大学 | A kind of image fusion system based on low-light and infrared sensor |
| CN109584197A (en) * | 2018-12-20 | 2019-04-05 | 广东浪潮大数据研究有限公司 | A kind of image interfusion method and relevant apparatus |
-
2018
- 2018-12-20 CN CN201811563609.4A patent/CN109584197A/en active Pending
-
2019
- 2019-08-30 WO PCT/CN2019/103638 patent/WO2020125062A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104200460A (en) * | 2014-08-04 | 2014-12-10 | 西安电子科技大学 | Image registration method based on images characteristics and mutual information |
| CN107095384A (en) * | 2017-04-26 | 2017-08-29 | 长春理工大学 | A kind of Intelligent fire-fighting helmet device transmitted based on WIFI |
| CN206712926U (en) * | 2017-05-26 | 2017-12-05 | 淮阴师范学院 | A kind of more exposure image emerging systems |
Non-Patent Citations (3)
| Title |
|---|
| JITENDRA R.RAOL 等著: "《移动智能自主系统》", 30 September 2018 * |
| 张美玲: "小波变换及其在图像融合中的应用", 《科技资讯》 * |
| 李然: "基于最大互信息的红外与可见光图像配准", 《电脑知识与技术》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020125062A1 (en) * | 2018-12-20 | 2020-06-25 | 广东浪潮大数据研究有限公司 | Image fusion method and related device |
| CN110633759A (en) * | 2019-09-24 | 2019-12-31 | 北京华力兴科技发展有限责任公司 | Image fusion method and device and electronic equipment |
| CN112272122A (en) * | 2020-10-14 | 2021-01-26 | 北京中科网威信息技术有限公司 | FPGA accelerator card detection method and device and readable storage medium |
| CN112819739A (en) * | 2021-01-28 | 2021-05-18 | 浙江祺跃科技有限公司 | Scanning electron microscope image processing method and system |
| CN112819739B (en) * | 2021-01-28 | 2024-03-01 | 浙江祺跃科技有限公司 | Image processing method and system for scanning electron microscope |
| CN115829897A (en) * | 2023-02-17 | 2023-03-21 | 湖北芯擎科技有限公司 | Image fusion processing method and device, electronic equipment and medium |
| CN115829897B (en) * | 2023-02-17 | 2023-06-06 | 湖北芯擎科技有限公司 | Image fusion processing method and device, electronic equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020125062A1 (en) | 2020-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111898696B (en) | Pseudo tag and tag prediction model generation method, device, medium and equipment | |
| CN109584197A (en) | A kind of image interfusion method and relevant apparatus | |
| CN112163634A (en) | Example segmentation model sample screening method and device, computer equipment and medium | |
| WO2021189901A1 (en) | Image segmentation method and apparatus, and electronic device and computer-readable storage medium | |
| Wu et al. | KD-PAR: A knowledge distillation-based pedestrian attribute recognition model with multi-label mixed feature learning network | |
| An et al. | Medical image segmentation algorithm based on multilayer boundary perception-self attention deep learning model | |
| EP3665614A1 (en) | Extraction of spatial-temporal features from a video | |
| Ntavelis et al. | AIM 2020 challenge on image extreme inpainting | |
| CN112419326B (en) | Image segmentation data processing method, device, equipment and storage medium | |
| Tang et al. | Stroke-based scene text erasing using synthetic data for training | |
| Wu et al. | Multi-features refinement and aggregation for medical brain segmentation | |
| Yu et al. | Multi-style image generation based on semantic image | |
| Yuan et al. | FGNet: Fixation guidance network for salient object detection | |
| Agarwal et al. | Unmasking the potential: evaluating image inpainting techniques for masked face reconstruction | |
| Mu et al. | Integration of gradient guidance and edge enhancement into super‐resolution for small object detection in aerial images | |
| CN114549322B (en) | Image super-resolution method and device based on self-adaption in unsupervised field | |
| CN116681824A (en) | Object surface material reconstruction method, device and storage medium | |
| Sun et al. | Ancient paintings inpainting based on dual encoders and contextual information | |
| Soni et al. | Multiencoder‐based federated intelligent deep learning model for brain tumor segmentation | |
| Wang et al. | Generating high-quality texture via panoramic feature aggregation for large mask inpainting | |
| Wang et al. | Research on texture image inpainting of jacquard fabric based on non-single vision | |
| Yang et al. | Res2U-Net: image inpainting via multi-scale backbone and channel attention | |
| Wang et al. | Image Semantic Segmentation Algorithm Based on Self-learning Super-Pixel Feature Extraction | |
| Li et al. | License plate detection and recognition technology for complex real scenarios | |
| Jia et al. | Mrbenet: A multiresolution boundary enhancement network for salient object detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190405 |
|
| RJ01 | Rejection of invention patent application after publication |