CN108288256A - Multispectral mosaic image restoration method - Google Patents
Multispectral mosaic image restoration method Download PDFInfo
- Publication number
- CN108288256A CN108288256A CN201810097052.3A CN201810097052A CN108288256A CN 108288256 A CN108288256 A CN 108288256A CN 201810097052 A CN201810097052 A CN 201810097052A CN 108288256 A CN108288256 A CN 108288256A
- Authority
- CN
- China
- Prior art keywords
- image
- pixel
- mosaic
- multispectral
- spectrum section
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000001228 spectrum Methods 0.000 claims abstract description 71
- 230000003595 spectral effect Effects 0.000 claims abstract description 64
- 238000005070 sampling Methods 0.000 claims abstract description 13
- 239000011159 matrix material Substances 0.000 claims abstract description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 238000011084 recovery Methods 0.000 claims description 5
- 241000287196 Asthenes Species 0.000 claims description 4
- 230000003716 rejuvenation Effects 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 8
- 238000003384 imaging method Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 6
- 238000011160 research Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 3
- 238000000701 chemical imaging Methods 0.000 description 2
- 238000000576 coating method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 241000894007 species Species 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- VMXUWOKSQNHOCA-UKTHLTGXSA-N ranitidine Chemical compound [O-][N+](=O)\C=C(/NC)NCCSCC1=CC=C(CN(C)C)O1 VMXUWOKSQNHOCA-UKTHLTGXSA-N 0.000 description 1
- 238000004611 spectroscopical analysis Methods 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a multispectral mosaic image restoration method, which can restore a multispectral mosaic video image shot by a film-coated video spectrometer into a complete multispectral image with high spatial resolution and high spectral resolution, and solves the application limitation caused by real-time transmission of the multispectral image, and the method comprises the following steps: 1) determining a mosaic template according to the number of spectral bands contained in a pixel matrix block of an original multispectral mosaic image S; 2) extracting a monospectrum image S1; 3) down-sampling the single-spectral-segment image S1 to obtain an image S2; 4) up-sampling the image S2 by 2 times to obtain an image S3: 5) performing interpolation operation on the image S3; 6) the restoration condition of the single spectrum segment image S1 is judged, and all the single spectrum segment image restoration is completed.
Description
Technical field
The present invention relates to a kind of multi-spectral image processing technologies, more particularly to multispectral mosaic image restored method.
Background technology
Super-resolution Reconstruction and rebuilding spectrum research and exploration in multi-spectral image processing are always multispectral image technology
One important content of application development.Multispectral image is a kind of high resolution remote sensing integrating image graphics and spectroscopy
Image, the spatial resolution and spectral resolution of image are all higher than common remote sensing images.Compared with ordinary numbers image, mostly light
Spectrogram picture has the spatial information and spectral information of atural object, and geometry information and the spectral characteristic that can obtain target simultaneously are bent
Line.But multispectral image, because its spectral coverage quantity is more, spatial resolution is higher, and it is big that there are data volumes, information redundance is high, data
The shortcomings of transmission rate is slow, making multispectral image, with equal tracks field, there is also applications to limit in real-time monitoring and mobile target.Cause
How this to ensure under conditions of the spatial resolution of multispectral image and constant spectral resolution, reduce information redundancy degree with
The technology for reaching multispectral image real-time data transmission and multispectral video imaging is a new research hotspot, numerous scientific research works
Author also throws oneself into this research.
For multispectral image data itself, common data characteristics mainly have：
1, spatial resolution is high, and spectral resolution is low, less suitable for ground species, and spatial resolution requirements are high to be held
Multispectral imaging, such as：Multispectral camera imaging technique.
2, spatial resolution is low, and spectral resolution is high, and more suitable for ground species, target geometry is big and regular
Airborne multispectral imaging, such as：Field-crop detects, forest monitoring etc..
3, spatial resolution is high, and spectral resolution is high, that is, high light spectrum image-forming technology, is suitable for aerospace scientific research and leads
Domain, such as：The high spectrum image of spaceborne hyperspectral imager shooting, is mainly used in atmospheric monitoring etc..
High, the data transmission speed that all inevitably there is data information redundancy for above-mentioned more than three kinds/high spectrum image
The shortcomings of rate is slow.This few multispectral figure image of class is mainly used in quiescent imaging scene at present, be not suitable for video imaging and
Mobile target shooting.Therefore the multi-optical spectrum image collecting mode of mosaic formation is applied to more/high spectrum image by scientific research scholar
In, by the way that filter coating is added before detector array, so that image is made of several picture element matrix blocks, as shown in Figure 1.
Similar color digital image imaging mode, picture element matrix different pixels point in the block respond different spectral coverage,
Each pixel in image only responds some spectral coverage information, and the multispectral image of output is similar to digital mosaic figure
Picture.This imaging mode can ensure to export the spatial resolution of image and spectral resolution is constant, and greatly reduces letter
Cease redundancy, real time transmitting image data, moreover it is possible to carry out multispectral image video capture.
But the image data of the multispectral video imaging instrument shooting of film coated type is used to be not directly applicable analyzing processing, because
By the multispectral image data compression of multidimensional it is two-dimensional data format for such imaging technique, the image of output has mosaic effect
It answers, the profile in image is relatively fuzzyyer, and the loss of detail of object is than more serious.Simple spectrum section image spatial resolution can be with spectral coverage number
The increase of amount and reduce, the spectral information of each pixel missing can also increase therewith, the situation especially more than the spectral coverage quantity
Under, this disadvantage can be more obvious.Therefore it needs to carry out restoration disposal to two-dimensional multispectral image data, makes multispectral image
Simple spectrum section image spatial resolution significantly increase, rebuild all spectral informations of each pixel, it is complete more to restore
Spectrum picture.
Invention content
In order to solve the problems in background technology, the present invention provides a kind of multispectral reconstructing restored sides of mosaic image
Method, can make to use the multispectral mosaic video image that film coated type video light spectrometer is shot to be reconditioned for high spatial resolution and
The complete multispectral image of high spectral resolution solves application limitation caused by multispectral image real-time Transmission.
The introduction of the principle of the present invention content：
A kind of multispectral mosaic image restored method is carried using mosaic Template Information from multispectral mosaic image
Simple spectrum section image is taken, the pixel value for lacking pixel in simple spectrum section image is calculated using Taylor series technique of estimation, rebuilds simple spectrum section figure
The spatially and spectrally information of picture makes the spatial resolution of simple spectrum section image as close possible to the resolution ratio of detector.Because calculating
The value for lacking pixel is that single image independently carries out, and avoids adjacent spectral segment information from interfering, the complete multispectral image light after reconstruction
Error is composed compared to other methods smaller.
Above application Taylor series technique of estimation is led using single order, the second order of known pixels point value around missing pixel point
Number, and Taylor series formula approximate evaluation missing pixel point value is utilized, the side of image can be greatly retained using the calculation
Edge information.By expanding known pixels point computer capacity in calculating process, it is serious sparse caused that image can be effectively reduced
Image detail information loss.
The above-mentioned complete multispectral image of reconstruction is built upon on the spatially and spectrally Information base of simple spectrum section image, and scaling carries
The original simple spectrum section image taken, completes the missing pixel values estimation operation of image after scaling, and repeats scaling and valuation operation
Journey, until the simple spectrum section image spatial resolution of reconstruction reaches the resolution ratio of original two dimensional image.By all simple spectrums after reconstruction
Section image restoration is complete multispectral image, i.e., reverts to multidimensional data from two-dimensional image data.
This method by scale process and Taylor series technique of estimation to two-dimentional multispectral image data carry out multiplanar reconstruction from
And complete multispectral image is restored, so that multispectral image data still is able to keep high spatial discrimination under the conditions of real-time Transmission
Rate and spectral resolution realize complete multispectral image video acquisition transmission, spectral technique are applied to movable object tracking
With detection.
The specific technical solution of the present invention is：
A kind of multispectral mosaic image restored method, includes the following steps：
1) the spectral coverage quantity contained in the picture element matrix block of original multispectral mosaic image S determines mosaic mould
Plate；If spectral coverage quantity is N, then mosaic template size is M × M, and N, M are positive integer, and M*M=N, N >=4；
2) extraction simple spectrum section image S1；
3) down-sampling is carried out to simple spectrum section image S1, i.e., the had sky pixel in image S1 is deleted, obtains image S2；
4) 2 times of up-samplings are carried out to image S2, obtains image S3：
5) interpolation arithmetic is carried out to image S3；
5.1) first time interpolation；
Interpolation arithmetic is carried out to the point of intersection of the image S3 empty pixels of each row of every a line being newly inserted into；
5.2) second of interpolation；
Empty pixel remaining to image S3 carries out interpolation arithmetic again, obtains image S4：The remaining empty pixel is located at same column two
The intersection position of the line of the line of a adjacent known pixel point and two adjacent known pixel points of going together；
6) judge the rejuvenating conditions of simple spectrum section image S1, and complete all simple spectrum section image restorations, specific practice is：
If the number for executing step 4) to step 5) during each simple spectrum section image restoration is P, P >=0；
When N is even number, and work asThen think that recovery is completed in the simple spectrum section image S1 of extraction, repeats step 2-
5) until all simple spectrum section images of original multispectral mosaic image all restore；
When N is even number, and work asThen continue to repeat step 4) to step 5), untilAgain to remaining simple spectrum
Section image executes step 2-5) until all simple spectrum section images of original multispectral mosaic image all restore；
When N is odd number, and work asAndStep 4) is then repeated to step 5), until full
Sufficient conditionObtained image S4, then be inserted into a line one every 2*P rows 2*P row in image S4 and arrange empty pixel, make figure
The size of picture be equal to simple spectrum section image S1, then to unknown pixel in the image after handling above using adjacent picture elements mean value into
Row valuation operation, then it is assumed that recovery is completed in the simple spectrum section image S1 of extraction, repeats step 2-5) until original multispectral horse
All simple spectrum section images for matching gram image all restore.
Above-mentioned steps 2) specific practice is：
Simple spectrum section image is extracted from original multispectral mosaic image S according to putting in order for spectral coverage in mosaic template
S1；
Specifically：Original multispectral mosaic image S is traversed using sliding window, the size of sliding window is equal to mosaic
It only is set to 1 there are one pixel in template size and sliding window, rest of pixels point is set to 0, and pixel is set to 1 position and horse
Position where having same spectral coverage information pixel in match gram template is the same；
Each small lattice represent a pixel in the simple spectrum section image S1, and each pixel responds a spectral coverage information and each
The spectral coverage information of a picture element matrix all pixel responses in the block is different.
Above-mentioned steps 3) specific practice is：
The pixel that pixel in simple spectrum section image S1 is set to 0 is deleted, and the spatial resolution of simple spectrum section image S1 can be reduced to
The 1/N of original multispectral mosaic image S, to obtain image S2.
Above-mentioned steps 4) specific practice is：
A row are inserted into a line one and arrange empty pixel in every line in image S2, to obtain image S3, wherein image S3's
Spatial resolution is 4 times of image S2.
Above-mentioned steps 5.1) and 5.2) in the detailed process of interpolation arithmetic be：
A1, P points are set as unknown pixel point, four known pixel neighborhood N around P points1, N2, N3, N4, have in each neighborhood
4 known pixels calculate the single order and two of the known pixel in each neighborhood using First-order Gradient operator and second order gradient operator
Order derivative；
A2, the single order and second dervative average value for calculating four known pixels in each neighborhood；
The first derivative mean value of four neighborhoods is in windowSecond dervative is equal
Value is
P points Taylor series approximation value on four neighborhood directions in A3, calculation window：
A4, the weight coefficient ω for calculating each neighborhoodi：(i=1,2,3,
4), (k=1,2,3,4)；ρ is constant, makes the weight coefficient ω of four neighborhoodsiThe sum of be 1；
A5, the estimated value I (P) for calculating P point pixels：
The beneficial effects of the invention are as follows：
1, the present invention is based on the mosaic multispectral image of film coated type video light spectrometer shooting, the mosaic mould of plated film is utilized
Plate and the Taylor series approximation estimation technique propose that a kind of multispectral mosaic image restores, the method for rebuilding complete multispectral image,
This method can greatly reduce the spectral information redundancy of image data using the video light spectrometer of film coated type, real-time transmission data,
But it will also result in that simple spectrum section image spatial resolution is too low, the spectral information serious loss of single pixel.And figure through the invention
As restored method, the space Super-resolution Reconstruction of simple spectrum section image can be carried out in data preprocessing phase and pixel lacks spectrum weight
It builds, realizes the perfect reconstruction of multispectral image video.
2, method of the invention solves multispectral mosaic image and causes simple spectrum section image because of the increase of spectral coverage quantity
Spatial discrimination reduces problem；It is insufficient to solve spectral information present in multispectral mosaic image, the imperfect problem of image data；
It is low to solve simple spectrum section image spatial resolution in multispectral mosaic image, spectral classification caused by spectral information missing is serious
The big problem of error；Make the complete multispectral image spatial resolution after reconstruction as close possible to detector resolution.
3, due to the spectral coverage quantity increase with spectrometer plated film, the simple spectrum section image spatial resolution of extraction can drop therewith
It is low, and the spectral coverage information of single pixel missing can also increase, and this can all cause simple spectrum section picture signal seriously sparse, after making reconstruction
Soft edge, spectral classification error increase.Therefore the present invention is used expands pixel to be estimated in simple spectrum section image
The range in surrounding known pixels domain is calculated using the single order of all known pixels points in neighborhood of pixel points to be estimated and second dervative
Taylor series approximation value, and by weighted formula calculate pixel to be inserted the spectral coverage pixel value.By expanding known pixels
Contiguous range and calculate single order, second dervative and can greatly retain the marginal information in original image, make the figure after reconstruction
Image space resolution ratio enhancing, spectral error rate reduce.
Description of the drawings
Fig. 1 is in the embodiment of the present invention using the original multispectral mosaic image of film coated type video light spectrometer shooting；
Fig. 2 is the plated film mosaic template used in the embodiment of the present invention
Fig. 3 is the simple spectrum section image extracted in the embodiment of the present invention；
Fig. 4 is the step of reconstruction to simple spectrum section image in the embodiment of the present invention；
Fig. 5 (a) is an interpolation point distribution schematic diagram in first time Interpolation Process in the embodiment of the present invention；
Fig. 5 (b) is all interpolation point distribution schematic diagrams in first time Interpolation Process in the embodiment of the present invention；
Fig. 5 (c) is an interpolation point distribution schematic diagram in second of Interpolation Process in the embodiment of the present invention；
Fig. 5 (d) is all interpolation point distribution schematic diagrams in second of Interpolation Process in the embodiment of the present invention；
Fig. 6 is the Taylor series estimation method with the non-non- same column pixel of going together of known pixel in the embodiment of the present invention；
Fig. 7 is different lines pixel Taylor series valuation of not going together or go together with known pixel same column in the embodiment of the present invention
Method；
Fig. 8 (a) is the multispectral mosaic image of original two dimensional of shooting；
Fig. 8 (b) is the partial enlarged view of 8 (a)；
Fig. 8 (c) is the partial enlarged view of 8 (b)；
Fig. 9 (a) is the simple spectrum section image after the method for the present invention is restored；
Fig. 9 (b) is the partial enlarged view of 9 (a)；
Fig. 9 (c) is the partial enlarged view of 9 (b).
Specific implementation mode
As shown in Figure 1, original multispectral mosaic image S is acquired with film coated type video light spectrometer, according to the plated film of camera lens
Type can select the imaging lens of 9 spectral coverages, 16 spectral coverages and 25 spectral coverages, and what is selected in the present embodiment is 25 spectral coverage mirror coatings
Head.Fig. 2 show the mosaic template (alternatively referred to as picture element matrix) of 25 spectral coverages, uses image that video light spectrometer shoots can be with
It is considered as and is made of several mosaic picture element matrixs, letter of each pixel only in response to a certain certain spectral in picture element matrix
Breath, and the information of each pixel response different spectral coverage.
As shown in Fig. 2, being extracted from original multispectral mosaic image S according to putting in order for spectral coverage in mosaic template
Simple spectrum section image S1 traverses original multispectral mosaic image S using sliding window, and the size of sliding window is equal to mosaic mould
Plate, only there are one points to be set to 1 in sliding window, remaining point is set to 0, be set to 1 point position and mosaic template in have same spectrum
Position where segment information pixel is the same.For example, the 3rd spectral coverage pixel present position of 25 spectral coverage mosaic templates is in Fig. 2
(1,3) then extracts the sliding window size that the 3rd spectral coverage image uses and there was only position (1,3) disposition for 5*5, and in window
It is 1, remaining point is set to 0.
As can be seen from Figure 3 the simple spectrum section image S1 spatial resolutions extracted decline seriously, only artwork image space point
The 1/25 of resolution.In order to improve the spatial resolution of simple spectrum section image, make the spatial resolution of all simple spectrum section images as far as possible
The resolution ratio of proximity detector, and make the spectral information of image after reconstruction as close possible to the spectral characteristic of real-world object, because
This is rebuild using such as the step of Fig. 4 in this example.
The first step：Down-sampling is carried out to it after extraction simple spectrum section image S1, obtains image S2；By the picture of void value in image
Member is deleted, and the 1/25 of the spatial resolution meeting dimensionality reduction original image of image；
Then 2 times of up-samplings are carried out to the image S2 after down-sampling, a row are inserted into empty pixel in every line in the picture, obtain
To image S3；Such as the second step during Fig. 3, the distribution of the known pixels point in image is made to be similar to chessboard.
Second step：First time interpolation
Taylor corrected series, pixel position such as Fig. 5 (a) of first time interpolation, (b) institute are carried out to the image S3 after down-sampling
Show (being exactly the point of intersection position to the image S3 empty pixels of each row of every a line being newly inserted into), white dot represents known picture
Member, grey dot represent the pixel that need to be inserted into this step, are inserted into pixel and known pixel is not gone together same column.To grey
Pixel calculating process at dot using 7*7 size sliding windows as shown in fig. 6, traverse entire image, to the center in window
Pixel carries out Taylor series approximation valuation calculating such as the P points in Fig. 6.Calculating process is：
Four known pixel neighborhood N around A1, selection P points1, N2, N3, N4, calculated using First-order Gradient operator and second order gradient
Son calculates the single order and second dervative of the known pixel in each neighborhood, such as：N2In neighborhood, four known pixel q21, q22, q23,
q24First derivative be I2′(q21), I2′(q22), I2′(q23), I2′(q24), second dervative I2″(q21), I2″(q22), I2″
(q23), I2″(q24)；
A2, the single order and second dervative average value for calculating four known pixels in each neighborhood, such as：N2In neighborhood, single order
Derivative mean valueSecond dervative mean valueFour neighbours in window
The first derivative mean value in domain is Second dervative mean value is
P points Taylor series approximation value on four neighborhood directions in A3, calculation window：,
A4, the weight coefficient ω for calculating each neighborhoodi：(i=1,2,3,
4), (k=1,2,3,4)；ρ is constant, makes the weight coefficient ω of four neighborhoodsiThe sum of be 1；
A5, the estimated value I (P) for calculating P point pixels：
Third walks：Second of interpolation
Valuation operation is carried out to remaining unknown pixel in image (shown in such as Fig. 5 (b)) after completion second step, it is remaining
Unknown pixel and known pixel go together different lines or same column is not gone together, and (dot of black represents quilt in this step in such as Fig. 5 (c)
The pixel of estimation is exactly the company for being located at the line of the adjacent known pixel point of same column two and two adjacent known pixel points of going together
The intersection position of line).So the neighborhood used in this step divides as shown in fig. 7, using four neighborhood N in Fig. 71, N2, N3, N4
Interior known image element information repeats the calculating process in second step, and the value by remaining unknown pixel in image in the spectral coverage calculates
Out, as a result as shown in Fig. 5 (d), black dot represents the pixel being inserted into third step.
4th step：Carry out 2 times of up-samplings to completing the image after third step, then to image repeat above second step and
The calculating process of third step, until image size is equal or close to original two dimensional image；
If the spectral coverage quantity in original two dimensional mosaic image is even number, only need to be answered according to above four steps
Original calculates resolution ratio of the image resolution ratio after capable of making recovery as close possible to detector.If in original two dimensional mosaic image
Spectral coverage quantity be odd number, then need to carry out following procedure after completing above-mentioned 4th step：
A, a line one is inserted into every certain even number line and even column arrange empty pixel in above-mentioned 4th step treated image,
Such as the 25 spectral coverage mosaic multispectral images used in the present invention carry out restoration disposal, then scheme after the processing of above-mentioned 4th step
Insertion a line one being arranged every four rows four as in and arranging empty pixel, the size of image is made to be equal to original two dimensional mosaic image.
B, valuation operation is carried out using the mean value of adjacent picture elements to unknown pixel in the image after handling above.
Above step is the restored method for simple spectrum section image, in order to restore complete multispectral image, is needed pair
Each width simple spectrum section image of extraction carries out above-mentioned all calculating process.It is more that 25 spectral coverages are shot using film coated type video light spectrometer
Spectral mosaic image, as shown in Fig. 8 (a), 8 (b), 8 (c).Restored method treated image such as Fig. 9 by the present invention
(a), 9 (b), 9 (c) are shown.The simple spectrum section image spatial resolution that can be seen that after processing from Contrast on effect in figure is shown
It writes and improves, the profile and detailed information of image are enhanced.
For restored method, it can ensure known pixels point value and position in such a way that down-sampling and up-sampling are combined
It is not influenced by interpolation, keeps the result of calculating more acurrate, the possibility that figure is distorted reduces.And utilize the Taylor for expanding neighborhood
Series interpolation method, image edge information can greatly be retained by calculating first derivative and second dervative, reduce spectral classification error.
Claims (5)
1. a kind of multispectral mosaic image restored method, which is characterized in that include the following steps：
1) the spectral coverage quantity contained in the picture element matrix block of original multispectral mosaic image S determines mosaic template；
If spectral coverage quantity is N, then mosaic template size is M × M, and N, M are positive integer, and M*M=N, N >=4；
2) extraction simple spectrum section image S1；
3) down-sampling is carried out to simple spectrum section image S1, i.e., the had sky pixel in image S1 is deleted, obtains image S2；
4) 2 times of up-samplings are carried out to image S2, obtains image S3：
5) interpolation arithmetic is carried out to image S3；
5.1) first time interpolation；
Interpolation arithmetic is carried out to the point of intersection of the image S3 empty pixels of each row of every a line being newly inserted into；
5.2) second of interpolation；
Empty pixel remaining to image S3 carries out interpolation arithmetic again, obtains image S4：The remaining empty pixel is located at two phases of same column
The intersection position of the line of the line of pixel point known to neighbour and two adjacent known pixel points of going together；
6) judge the rejuvenating conditions of simple spectrum section image S1, and complete all simple spectrum section image restorations, specific practice is：
If the number for executing step 4) to step 5) during each simple spectrum section image restoration is P, P >=0；
When N is even number, and work asThen think that recovery is completed in the simple spectrum section image S1 of extraction, repeat step 2-5) it is straight
All simple spectrum section images to original multispectral mosaic image all restore；
When N is even number, and work asThen continue to repeat step 4) to step 5), untilAgain to remaining simple spectrum section figure
As executing step 2-5) until all simple spectrum section images of original multispectral mosaic image all restore；
When N is odd number, and work asAndStep 4) is then repeated to step 5), until meeting item
PartObtained image S4, then be inserted into a line one every 2*P rows 2*P row in image S4 and arrange empty pixel, make image
Size is equal to simple spectrum section image S1, then is estimated using the mean value of adjacent picture elements to unknown pixel in the image after handling above
It is worth operation, then it is assumed that recovery is completed in the simple spectrum section image S1 of extraction, repeats step 2-5) until original multispectral mosaic
All simple spectrum section images of image all restore.
2. multispectral mosaic image restored method according to claim 1, it is characterised in that：The step 2) is specifically done
Method is：
Simple spectrum section image S1 is extracted from original multispectral mosaic image S according to putting in order for spectral coverage in mosaic template；
Specifically：Original multispectral mosaic image S is traversed using sliding window, the size of sliding window is equal to mosaic template
It only is set to 1 there are one pixel in size and sliding window, rest of pixels point is set to 0, and pixel is set to 1 position and mosaic
Position where having same spectral coverage information pixel in template is the same；
Each small lattice represent a pixel in the simple spectrum section image S1, and each pixel responds a spectral coverage information and each picture
The spectral coverage information of prime matrix all pixel responses in the block is different.
3. multispectral mosaic image restored method according to claim 1, it is characterised in that：The step 3) is specifically done
Method is：
The pixel that pixel in simple spectrum section image S1 is set to 0 is deleted, and the spatial resolution of simple spectrum section image S1 can be reduced to original
The 1/N of multispectral mosaic image S, to obtain image S2.
4. multispectral mosaic image restored method according to claim 1, it is characterised in that：The step 4) is specifically done
Method is：
A row are inserted into a line one and arrange empty pixel in every line in image S2, to obtain image S3, the wherein space of image S3
Resolution ratio is 4 times of image S2.
5. multispectral mosaic image restored method according to claim 1, it is characterised in that：The step 5.1) and
5.2) detailed process of interpolation arithmetic is in：
A1, P points are set as unknown pixel point, four known pixel neighborhood N around P points1, N2, N3, N4, have in each neighborhood 4
Know pixel, the single order and second order that the known pixel in each neighborhood is calculated using First-order Gradient operator and second order gradient operator are led
Number；
A2, the single order and second dervative average value for calculating four known pixels in each neighborhood；
The first derivative mean value of four neighborhoods is in windowSecond dervative mean value is
P points Taylor series approximation value on four neighborhood directions in A3, calculation window：
A4, the weight coefficient ω for calculating each neighborhoodi：(i=1,2,3,4), (k
=1,2,3,4)；ρ is constant, makes the weight coefficient ω of four neighborhoodsiThe sum of be 1；
A5, the estimated value I (P) for calculating P point pixels：
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810097052.3A CN108288256B (en) | 2018-01-31 | 2018-01-31 | Multispectral mosaic image restoration method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810097052.3A CN108288256B (en) | 2018-01-31 | 2018-01-31 | Multispectral mosaic image restoration method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108288256A true CN108288256A (en) | 2018-07-17 |
| CN108288256B CN108288256B (en) | 2020-07-31 |
Family
ID=62836170
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810097052.3A Active CN108288256B (en) | 2018-01-31 | 2018-01-31 | Multispectral mosaic image restoration method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108288256B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109029380A (en) * | 2018-09-14 | 2018-12-18 | 中国科学院西安光学精密机械研究所 | Stereoscopic vision system based on film-coated multispectral camera and calibration ranging method thereof |
| CN109146787A (en) * | 2018-08-15 | 2019-01-04 | 北京理工大学 | A kind of real-time reconstruction method of the double camera spectrum imaging system based on interpolation |
| CN110458766A (en) * | 2019-07-11 | 2019-11-15 | 天津大学 | A kind of fast illuminated high spectrum image demosaicing methods |
| CN110579279A (en) * | 2019-09-19 | 2019-12-17 | 西安理工大学 | design method of nine-spectral-band multispectral imaging system of single sensor |
| WO2021003594A1 (en) * | 2019-07-05 | 2021-01-14 | Baidu.Com Times Technology (Beijing) Co., Ltd. | Systems and methods for multispectral image demosaicking using deep panchromatic image guided residual interpolation |
| CN112288008A (en) * | 2020-10-29 | 2021-01-29 | 四川九洲电器集团有限责任公司 | Mosaic multispectral image disguised target detection method based on deep learning |
| CN112489042A (en) * | 2020-12-21 | 2021-03-12 | 大连工业大学 | Metal product printing defect and surface damage detection method based on super-resolution reconstruction |
| CN114397255A (en) * | 2021-11-12 | 2022-04-26 | 中国科学院西安光学精密机械研究所 | Wide-spectrum high-resolution video spectral imaging system and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221128A (en) * | 2007-04-18 | 2008-07-16 | 中国科学院自动化研究所 | Multi-optical spectrum reconstruction method based on self-adapting finite element |
| CN103793883A (en) * | 2013-12-11 | 2014-05-14 | 北京工业大学 | Principal component analysis-based imaging spectral image super resolution restoration method |
| CN105469360A (en) * | 2015-12-25 | 2016-04-06 | 西北工业大学 | Non local joint sparse representation based hyperspectral image super-resolution reconstruction method |
| CN106780423A (en) * | 2017-01-12 | 2017-05-31 | 清华大学 | It is a kind of based on a small number of wave bands partial image high and the low point of high-quality spectrum reconstruction method of high spectrum image |
-
2018
- 2018-01-31 CN CN201810097052.3A patent/CN108288256B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221128A (en) * | 2007-04-18 | 2008-07-16 | 中国科学院自动化研究所 | Multi-optical spectrum reconstruction method based on self-adapting finite element |
| CN103793883A (en) * | 2013-12-11 | 2014-05-14 | 北京工业大学 | Principal component analysis-based imaging spectral image super resolution restoration method |
| CN105469360A (en) * | 2015-12-25 | 2016-04-06 | 西北工业大学 | Non local joint sparse representation based hyperspectral image super-resolution reconstruction method |
| CN106780423A (en) * | 2017-01-12 | 2017-05-31 | 清华大学 | It is a kind of based on a small number of wave bands partial image high and the low point of high-quality spectrum reconstruction method of high spectrum image |
Non-Patent Citations (1)
| Title |
|---|
| JIATONG HAN 等: ""Taylor series-based generic demosaicking algorithm for multispectral image"", 《PROC. SPIE 10462, AOPC 2017: OPTICAL SENSING AND IMAGING TECHNOLOGY AND APPLICATIONS》 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109146787A (en) * | 2018-08-15 | 2019-01-04 | 北京理工大学 | A kind of real-time reconstruction method of the double camera spectrum imaging system based on interpolation |
| CN109146787B (en) * | 2018-08-15 | 2022-09-06 | 北京理工大学 | Real-time reconstruction method of dual-camera spectral imaging system based on interpolation |
| CN109029380A (en) * | 2018-09-14 | 2018-12-18 | 中国科学院西安光学精密机械研究所 | Stereoscopic vision system based on film-coated multispectral camera and calibration ranging method thereof |
| CN109029380B (en) * | 2018-09-14 | 2019-12-03 | 中国科学院西安光学精密机械研究所 | Stereoscopic vision system based on film-coated multispectral camera and calibration ranging method thereof |
| CN112805744A (en) * | 2019-07-05 | 2021-05-14 | 百度时代网络技术（北京）有限公司 | System and method for demosaicing multispectral images using depth panchromatic image-guided residual interpolation |
| WO2021003594A1 (en) * | 2019-07-05 | 2021-01-14 | Baidu.Com Times Technology (Beijing) Co., Ltd. | Systems and methods for multispectral image demosaicking using deep panchromatic image guided residual interpolation |
| US11410273B2 (en) | 2019-07-05 | 2022-08-09 | Baidu Usa Llc | Systems and methods for multispectral image demosaicking using deep panchromatic image guided residual interpolation |
| CN112805744B (en) * | 2019-07-05 | 2024-04-09 | 百度时代网络技术（北京）有限公司 | System and method for demosaicing multispectral images |
| CN110458766A (en) * | 2019-07-11 | 2019-11-15 | 天津大学 | A kind of fast illuminated high spectrum image demosaicing methods |
| CN110458766B (en) * | 2019-07-11 | 2023-08-25 | 天津大学 | Snapshot hyperspectral image demosaicing method |
| CN110579279A (en) * | 2019-09-19 | 2019-12-17 | 西安理工大学 | design method of nine-spectral-band multispectral imaging system of single sensor |
| CN110579279B (en) * | 2019-09-19 | 2021-08-06 | 西安理工大学 | Design method of nine-spectral-band multispectral imaging system of single sensor |
| CN112288008A (en) * | 2020-10-29 | 2021-01-29 | 四川九洲电器集团有限责任公司 | Mosaic multispectral image disguised target detection method based on deep learning |
| CN112489042A (en) * | 2020-12-21 | 2021-03-12 | 大连工业大学 | Metal product printing defect and surface damage detection method based on super-resolution reconstruction |
| CN114397255A (en) * | 2021-11-12 | 2022-04-26 | 中国科学院西安光学精密机械研究所 | Wide-spectrum high-resolution video spectral imaging system and method |
| CN114397255B (en) * | 2021-11-12 | 2023-09-01 | 中国科学院西安光学精密机械研究所 | Wide-spectrum high-resolution video spectrum imaging system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108288256B (en) | 2020-07-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108288256A (en) | Multispectral mosaic image restoration method | |
| Deng et al. | Machine learning in pansharpening: A benchmark, from shallow to deep networks | |
| Shao et al. | Remote sensing image fusion with deep convolutional neural network | |
| CN110119780B (en) | Hyper-spectral image super-resolution reconstruction method based on generation countermeasure network | |
| CN108830796B (en) | Hyperspectral image super-resolution reconstruction method based on spectral-spatial combination and gradient domain loss | |
| Cheng et al. | Inpainting for remotely sensed images with a multichannel nonlocal total variation model | |
| Xie et al. | Hyperspectral image super-resolution using deep feature matrix factorization | |
| Liebel et al. | Single-image super resolution for multispectral remote sensing data using convolutional neural networks | |
| CN112805744B (en) | System and method for demosaicing multispectral images | |
| González-Audícana et al. | A low computational-cost method to fuse IKONOS images using the spectral response function of its sensors | |
| Zhang et al. | LR-Net: Low-rank spatial-spectral network for hyperspectral image denoising | |
| CN110501072B (en) | Reconstruction method of snapshot type spectral imaging system based on tensor low-rank constraint | |
| CN109146787B (en) | Real-time reconstruction method of dual-camera spectral imaging system based on interpolation | |
| CN106920214B (en) | Super-resolution reconstruction method for space target image | |
| CN111127374A (en) | Pan-sharing method based on multi-scale dense network | |
| Li et al. | Drcr net: Dense residual channel re-calibration network with non-local purification for spectral super resolution | |
| US9336570B2 (en) | Demosaicking system and method for color array based multi-spectral sensors | |
| CN110490924B (en) | Light field image feature point detection method based on multi-scale Harris | |
| CN111339989A (en) | Water body extraction method, device, equipment and storage medium | |
| CN109360147A (en) | Multispectral image super resolution ratio reconstruction method based on Color Image Fusion | |
| CN109977834B (en) | Method and device for segmenting human hand and interactive object from depth image | |
| CN114511470A (en) | Attention mechanism-based double-branch panchromatic sharpening method | |
| Chen et al. | Scene segmentation of remotely sensed images with data augmentation using U-net++ | |
| Castrodad et al. | Discriminative sparse representations in hyperspectral imagery | |
| CN114092327B (en) | Hyperspectral image super-resolution method utilizing heterogeneous knowledge distillation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |