CN106909902A - A kind of remote sensing target detection method based on the notable model of improved stratification - Google Patents
A kind of remote sensing target detection method based on the notable model of improved stratification Download PDFInfo
- Publication number
- CN106909902A CN106909902A CN201710115840.6A CN201710115840A CN106909902A CN 106909902 A CN106909902 A CN 106909902A CN 201710115840 A CN201710115840 A CN 201710115840A CN 106909902 A CN106909902 A CN 106909902A
- Authority
- CN
- China
- Prior art keywords
- target
- layer
- feature
- image
- super
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 93
- 238000013517 stratification Methods 0.000 title abstract 2
- 230000011218 segmentation Effects 0.000 claims abstract description 48
- 230000004927 fusion Effects 0.000 claims abstract description 20
- 230000003044 adaptive effect Effects 0.000 claims abstract description 6
- 230000008713 feedback mechanism Effects 0.000 claims abstract description 6
- 238000000034 method Methods 0.000 claims description 88
- 230000008569 process Effects 0.000 claims description 60
- 238000010586 diagram Methods 0.000 claims description 39
- 238000004422 calculation algorithm Methods 0.000 claims description 21
- 239000011159 matrix material Substances 0.000 claims description 21
- 238000012549 training Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 9
- 230000002787 reinforcement Effects 0.000 claims description 9
- 239000000654 additive Substances 0.000 claims description 6
- 230000000996 additive effect Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 230000001629 suppression Effects 0.000 claims description 5
- 239000013598 vector Substances 0.000 claims description 5
- 230000007246 mechanism Effects 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 238000007499 fusion processing Methods 0.000 claims description 3
- 238000003064 k means clustering Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000012887 quadratic function Methods 0.000 claims description 3
- 230000000717 retained effect Effects 0.000 claims description 3
- 238000011524 similarity measure Methods 0.000 claims description 3
- 238000011426 transformation method Methods 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 2
- 238000010276 construction Methods 0.000 abstract 1
- 239000003623 enhancer Substances 0.000 abstract 1
- 238000005286 illumination Methods 0.000 description 10
- 230000000694 effects Effects 0.000 description 9
- 230000008859 change Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 239000003086 colorant Substances 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of remote sensing target detection method based on the notable model of improved stratification, its step is:1. input picture is carried out into super-pixel segmentation;2. the low-level image feature of each super-pixel in input picture, construction global information collection and background information collection are extracted;3. learn the characteristic similarity between each super-pixel and background information collection, extract potential target characteristic pattern;4. the target signature of airport and oil tank target is generated;5. fusion potential target characteristic pattern and target signature generate level notable figure;6. the end criterion of adaptive learning is defined using LDA, if meet condition performing 8., is otherwise performed 7.;7. using the feedback mechanism between level, current layer input picture is strengthened using level notable figure as enhancer, using enhanced image as the next layer of input picture of study, performs the study for 1. starting new one layer;8. using the level notable figure of step 5. current layer as final notable figure, determine target location and class label, complete remote sensing target detection.
Description
Technical Field
The invention belongs to the application field of remote sensing image processing and scene understanding, and relates to a method for simultaneously detecting a plurality of interested remote sensing targets in a low-resolution remote sensing image by using an improved hierarchical reinforcement learning model.
Background
With the rapid development of high-resolution remote sensing satellite application technology, the typical remote sensing target detection technology has a crucial meaning in both military and civil fields. Particularly in the military field, the target position information can be accurately and quickly obtained from a large-format remote sensing image with a complex background, and technical support is provided for accurate guidance. Under the promotion of rapid development of image processing and pattern recognition theories, deep mining and intelligent processing of rich information of remote sensing images become a research hotspot and difficulty. The remote sensing image has rich information and large data volume, and not only comprises complicated and changeable ground environments and artificial targets with different shapes, but also influences image quality by blurring caused by uneven illumination and cloud layer shielding and target deformation and distortion caused by atmospheric refraction and atmospheric turbulence. The remote sensing image acquisition process is also influenced by imaging equipment and weather, when illumination changes and is interfered by rain, snow, smoke and dense fog, the definition of the acquired remote sensing image is reduced, the refraction caused by accumulated water can change the information such as appearance and texture of a target, and further change the characteristics such as shape and gray scale of the target, meanwhile, when the illumination changes, important information of the target is lost due to excessive or insufficient exposure of the camera equipment, the detection of the target can be greatly interfered, and even the detection fails. In addition, the remote sensing images have various target shapes, different colors and variable structures, and also interfere detection, targets are distributed in two dimensions on the space except bridges, airports and the like, most of the other targets are in a three-dimensional state and are represented as shadows of the targets, and the difficulty of target detection is increased. The shape of the target in the remote sensing image is incomplete due to the fact that the target is shielded by vegetation or other obstacles, and the position of the target is uncertain due to the fact that the aerial image is acquired randomly. Since the complex external environment brings difficulty to the target detection in the remote sensing image, the adoption of a proper method to detect the remote sensing target from the remote sensing image with complex background, various environments and rich information content becomes the key point of research of many scholars.
The current detection and identification algorithm of a typical target mainly comprises the following steps: clustering-based methods, feature matching-based methods, classifier-based methods, and the like. The clustering-based method is an unsupervised algorithm, although the method does not need to train samples, the time for manual labeling and sample training is reduced, but due to the fact that target distortion and deformation exist in the remote sensing image, false detection and missed detection can be caused by the unsupervised method, the method has great limitation, and the method is difficult to be widely applied to target detection of the remote sensing image. The feature matching based method usually uses low-level features such as texture features, local feature descriptors and the like to match with template features to realize detection, and has large computation amount and poor self-adaptive capability. The classifier adopted by the classifier-based method mainly comprises a Support Vector Machine (SVM), a feature bag model, an Adaboost classifier, a neural network classifier and the like. These methods firstly require a sufficient number of training samples to train the classifier, secondly require selecting suitable and effective features to classify, and require manual calibration of a large number of sample labels, which is a significant time cost.
The patent CN103729848A provides a hyperspectral remote sensing image small target detection method based on spectral saliency, the method selects spectral information and spatial information of an image to construct a feature vector, an improved Itti model and an improved evolution planning method are applied to a local saliency map and a global saliency map, and a total visual saliency map is finally generated to serve as a final target detection result. The method can only detect the approximate position of the target area, but cannot obtain the accurate boundary information of the target, and has poor adaptability to the target detection problem under the complex background.
The patent CN102214298A proposes a method for rapidly detecting and identifying an airport target based on a remote sensing image of a selective visual attention mechanism, which utilizes an improved attention selection model (GBVS) to obtain a salient region of the remote sensing image, and then combines an HDR tree according to SIFT characteristics on the region to achieve the purpose of identifying the airport target. Patent CN104156722A proposes an airport target detection method based on high-resolution remote sensing images, which detects parallel straight lines in images as airport runways, and has poor robustness to uncertain factors such as distortion and occlusion in the image shooting process.
Patent 201610247053.2 proposes a saliency detection model based on hierarchical reinforcement learning for detecting airport objects in low resolution remote sensing images. However, the method can only complete the detection task of a single target aiming at the airport target, and has no effect on the detection problem of other remote sensing targets, so on the basis of the patent, the invention provides an improved remote sensing target detection method of a hierarchical significant model, and the accurate detection of the airport and oil tank targets can be realized in the remote sensing image with a complex background.
Disclosure of Invention
The invention solves the problems: aiming at the problem of detecting the remote sensing target in the low-resolution remote sensing image, the improved method for detecting the remote sensing target of the hierarchical significant model is provided, and the problem of detecting the target in the remote sensing image under the condition of large breadth and low resolution can be accurately and efficiently solved.
The technical scheme of the invention is as follows: a remote sensing target detection method based on an improved hierarchical significant model mainly aims at two types of targets, namely an airport target and an oil tank target, and comprises the following steps:
step 1: performing superpixel segmentation on an input image to be detected by using a Simple linear iterative Clustering algorithm (SLIC), Clustering pixels with color similarity in a neighboring region in the image, and representing the regions clustered into a class by using superpixels so as to obtain an image subjected to superpixel segmentation;
step 2: taking the color feature of each superpixel in the segmented image as a bottom layer feature, constructing a global information set, and simultaneously extracting the bottom layer features of all superpixels positioned at the boundary position of the segmented image to construct a background information set;
and step 3: learning the feature similarity between each super pixel and a background information set by adopting a minimum Distance-based similarity measurement operator LDSM (LDSM), wherein the feature similarity is represented by a learning coefficient of the LDSM, the value of the learning coefficient is in positive correlation with the feature similarity between each super pixel and the background information set, and a potential target feature map is constructed by utilizing the learning coefficient and can reflect the feature difference between each super pixel region and the background information set and comprises all candidate significant target regions;
and 4, step 4: extracting prior characteristics of an airport target and an oil tank target from top to bottom from an input image to be detected, and respectively generating target characteristic graphs; for an airport target, performing linear detection on an input image to be detected by using a Linear Segment Detector (LSD) to obtain a linear detection result; then, counting the number of pixel points on the detected straight line in each super pixel region by using the image obtained by super pixel segmentation in the step 1, and generating a straight line density graph as a target characteristic graph of the airport target; for an oil tank target, carrying out circle detection on an input image to be detected by using a Hough transformation method to obtain a circle detection result, distributing different weights to the detected circle internal points and non-internal points by using a voting mechanism to generate a circle feature map, and taking the circle feature map as a target feature map of the oil tank target;
and 5: fusing the potential target feature map in the step 3 with the airport target feature map and the oil tank target feature map in the step 4, namely three images, and generating a level saliency map; the salient regions in the hierarchical salient map comprise candidate salient target regions, airport target regions and oil tank target regions in the potential target feature map obtained by each layer in the learning process;
step 6: defining an adaptive learning ending criterion by using a Latent topic semantic model LDA (LDA), and judging the feature similarity between the salient region in the current-level salient map in the step 5 and the airport and the oil tank target by using the criterion so as to judge whether the current learning process should be ended, and if the learning process is not ended, executing a step 7; if the learning process has ended, go to step 8;
and 7: feeding the level saliency map obtained from each layer back to the input image of the current layer by adopting a multilayer learning frame and utilizing a feedback mechanism between adjacent layers to realize background suppression layer by layer, namely taking the level saliency map obtained through fusion in the step 5 as an enhancement factor to act on the input image of the current layer, enhancing the image of the current layer after superpixel segmentation, taking a new image obtained after enhancement as the input image of the next layer for learning, executing the step 1 and starting the learning process of a new layer; repeating the steps, and gradually highlighting the target area through multi-layer learning;
and 8: and after the learning is stopped, taking the level saliency map obtained in the step 5 in the current layer as a final saliency map, extracting a saliency region in the final saliency map as a target region, and marking the target region and a category label of the target, namely an airport target or an oil tank target, in the image to be detected, so as to obtain a final result of remote sensing target detection.
In the step 1, the specific steps of performing super-pixel segmentation by using the SLIC algorithm are as follows:
the method comprises the steps of taking color features of a color image and position information of each pixel point as constraints for an input image I to be detected, clustering by adopting a K-means clustering algorithm, extracting color features of an image CIELab (Commission International Del' Eclairage), wherein the CIELab is a color system of the CIE, and coordinates of the color space are respectively L, a and b, so that the CIELab color space is called), representing local pixel points with similar color features by superpixels, thereby completing a superpixel segmentation process of the image, selecting the total number of superpixels segmented in the whole image as K, and enabling the segmented image I to comprise K superpixel regions.
In step 2, the specific steps of constructing the global information set and the background information set are as follows:
(1) knowing that the total number of superpixels contained in the image after superpixel segmentation is k, selecting the color feature of each superpixel to construct the bottom layer feature, namely converting the image into CIELab color space, respectively solving the values of all pixels in each superpixel in three color channels of the CIELab color space, and taking the mean value of all pixel points in each superpixel in each color channel as the bottom layer feature of the superpixel, namely the bottom layer feature p of the ith superpixeliExpressed as:
pi=(lli,lai,lbi) (1)
wherein, lli,lai,lbiRespectively representing the average value of all pixel points in the ith super pixel in the CIELab color space in the values of the three color channels, and then the feature set of all super pixels is P ═ { P ═1,p2,…,pkDefining P as a global information set; wherein p is1,p2,…,pkRespectively representing the bottom layer characteristics of 1 st, 2 nd, … th k super pixel areas in the segmented image; i is more than or equal to 1 and less than or equal to k, i represents any super pixel in the global information set, and k represents the total number of the super pixels;
(2) for a global information set P ═ P1,p2,…,pkN super-pixels are arranged on the image boundary, the n super-pixels on the boundary are extracted to form a background information set, n represents the total number of the background super-pixels, and a background information set B represents:
B={b1,b2,…,bn},0<n<k (2)
bj=(llj,laj,lbj),1≤j≤n (3)
where j represents any super-pixel in the background information set, bjUnderlying features, ll, representing super-pixel regions in a background information setj,laj,lbjRespectively representing the average values of all pixel points in the jth super pixel of the CIELab color space in the values of the three color channels.
In the step 3, a potential target feature map is constructed by using the learning coefficient, which specifically includes the following steps:
defining a superpixel dataset (p)i,bj) Wherein p isiBottom layer features representing super-pixel regions in the global feature set, bjRepresenting an underlying feature of a super-pixel region in a background information set, i.e. pi∈P,bj∈ B, the LDSM operator is used to obtain the bottom characteristic p of the super-pixel area in each global information setiWith the underlying features b of the super-pixel region in each background information setjCoefficient of similarity between αijAs shown in the following formula:
by solving equation (4), the learning coefficient α of the similarity measure is obtainedijWhen p isiAnd bjWhen they are close, αijIs approximately 1; when p isiAnd bjWhen they are completely equal αijEqual to 1; when p isiAnd bjWhen the difference is large, αijAway from 1; due to piAnd bjIs expressed as αijDegree of deviation from 1, the similarity learning coefficient αijNormalized to β according to equation (5)ijThen, then
Wherein,represents will | aij-1| is normalized to the interval [0,1 |)]Normalized learning coefficient βijThe closer to 0, the lower level feature p representing the global superpixel regioniBottom layer feature b closer to background super pixel areaj;
By solving the above optimization problem, a set of normalized learning coefficients is obtained, as shown in the following equation:
i represents any row of the matrix in equation (6), the ith row represents the learning coefficient obtained by learning all background superpixels by the ith superpixel in the feature set, the potential features β are defined by taking the mean value of each element in each rowiThe calculation formula of (2) is as follows:
where n is the total background superpixel number, βijRepresenting any one element in the matrix of the formula (6), and obtaining a potential target feature map F (β) from the formula (7)1,β2,…,βk) And k represents the total number of superpixels.
In the step 4, the target feature maps of the airport and oil tank targets are extracted, and the following steps are realized:
(1) extracting airport target characteristic diagram
For the input image I to be detected, LSD straight line detection is appliedThe measuring operator preliminarily obtains the straight line information of the whole image, then the straight line density characteristic is calculated corresponding to the super-pixel segmentation result of the current layer, the numerator is the number of pixel points on the straight line in each super-pixel region, and the denominator is the area of the corresponding region. The corresponding linear density characteristic d of any super pixel region i is expressed by the formula (8)iComprises the following steps:
wherein NumLRepresenting the number of pixel points on a straight line, Num is the total number of pixels, region (i) represents the ith super pixel region, and therefore, an object feature map based on the airport object is obtained, namely a straight line density map D ═ D1,d2,…,dk) Wherein d is1,d2,…,dkRespectively representing the linear density characteristics of 1, 2, … th superpixel areas, wherein k represents the total number of superpixels;
(2) extracting oil tank target characteristic diagram
The circle feature is a typical regional feature of an oil tank target, circle detection is carried out on an input image to be detected by utilizing Hough transformation, then a circle detection result is converted into a feature diagram, and the feature diagram is used as an oil tank target feature diagram;
when circle detection is carried out by utilizing Hough transformation, voting is carried out on (a, c, r) consisting of circle center positions (a, c) and circle radius r, wherein a is a circle center abscissa and c is a circle center ordinate; the local peak value in the voting result is the existing center position coordinate and radius of the circle, the voting result is used as the weight value of the circle feature, and all points meeting the formula (9) should share the same weight value as the corresponding (a, c, r), that is, all points should share the same weight value as the corresponding (a, c, r), that is, the local peak value in the voting result is the existing center position coordinate and radius of the circle
(x-a)2+(y-c)2≤r2(9)
Wherein (x, y) is the coordinate of any position in the target characteristic diagram, and the original weight values of other points in the non-circle are retained to obtain a target characteristic diagram C of the oil tank target.
The step 5 of fusing and generating the level saliency map specifically comprises the following steps:
firstly, carrying out additive fusion on a target characteristic diagram D of an airport target and a target characteristic diagram C of an oil tank target, namely performing union operation to obtain an image subjected to additive fusion; and then fusing the potential target feature graph F and the image after the addition fusion again, wherein the fusion process adopts a mode of multiplication among corresponding pixel points, namely intersection operation is taken, and the fusion mode is expressed as follows:
S=(D+C)·F (10)
wherein, S represents a saliency map generated through fusion in each layer learning process, i.e. a hierarchical saliency map.
In step 6, defining an adaptive learning termination criterion, and determining whether to terminate the current learning process, specifically including:
(1) firstly, obtaining sample images of an airport target and an oil tank target from a database, respectively training an LDA model by using the sample of the airport target and the sample of the oil tank target to obtain a theme model of the airport target and the oil tank target, and simultaneously training a theme model of a background to obtain the theme model of the airport target, the oil tank target and the background. In the LDA model training process, the characteristics of all training samples are selected as the color characteristics of a CIELab color space; after LDA model training, a background topic model p (z | bg) and an airport topic model p (z | fg) are obtained1) And tank topic model p (z | fg)2);
(2) For the level saliency map of each layer, a threshold of 0.6 is set. When the value of the super pixel area is more than 0.6, the super pixel area is considered as a significant area, otherwise, the super pixel area is considered as not significant; computing a topic model p (zs) for the ith salient regioni) Wherein s isiRepresenting the ith super-pixel salient region, a topic model p (z | s) is calculatedi) P (z | bg), p (z | fg)1),p(z|fg2) The type label corresponding to the topic model with the minimum distance is taken as the distance between the topic models, and the type labels are classifiedOne of an airport target, a tank target or a background is given to the salient region siLabeling the same labels, finishing learning when the type labels of all the salient regions are the type labels of the targets, and outputting the position of each salient region and the type label value of the salient region, otherwise, continuing the reinforcement learning process of the next layer; the cosine distance between the two vectors is used when calculating the distance between the topic models.
In the step 7, a multi-layer learning framework is adopted, and a feedback mechanism is utilized between adjacent layers to feed back the level saliency map of each layer to the input image of the current layer, so as to realize background suppression layer by layer, specifically comprising:
when the learned hierarchical saliency map S of the current layer does not satisfy the learning termination condition, the hierarchical saliency map S is first stretched by a stretching function, which is as follows:
R=f(S) (11)
wherein f represents a stretching function applied to the hierarchical saliency map S, said stretching function selecting a quadratic function; and defining the stretched matrix R as an enhancement matrix, and applying the enhancement matrix to enhance the input image. The image enhancement formula for the first layer is as follows:
I2=I·R1(12)
R1is an enhanced matrix of a first layer in the learning process, and an enhanced image I of the first layer is obtained by the feedback between layers of an input image I to be detected2Is shown by2As input image for the second layer reinforcement learning process, pair I2And (3) performing the characteristic extraction step to obtain a level saliency map of the second-level learning process, and by analogy, the feedback expression among levels is as follows:
It+1=It·Rt(13)
wherein, ItAnd It+1Representing input images of the t-th and t + 1-th layers, respectively, RtThe enhancement matrix for the t-th layer, the layer by equation (13)Inter-stage feedback Process, input image I of layer t +1t+1The saliency of the target area is preserved while the background area is suppressed;
in the t-th learning process, the input image I of the current layertPerforming superpixel segmentation on the image by using SLIC algorithm, and segmenting the image into ktA super pixel region; the adopted super-pixel segmentation number meets the following conditions:
k1≥k2≥k3≥…≥kt≥…,t=1,2,3,… (14)
namely, a segmentation mode from fine to coarse is adopted, the edge characteristics of the image can be accurately reserved by the initial fine segmentation, and the calculation amount can be properly reduced by the subsequent coarse segmentation;
updated image It+1The learning of a new layer is started with the input image of the next layer, and step 1 is executed.
In step 8, obtaining the target area and the category label of the target in the final saliency map specifically includes:
in the learning process, if the level saliency map of the current layer meets the self-adaptive learning ending criterion, ending the learning process; the number T of layers where the current layer is located is represented as the total number of learning layers and is also the iteration number of the learning process; taking the hierarchical saliency map of the current layer as a final saliency map, namely a final saliency map SfinalExpressed as:
Sfinal=ST(15)
STa hierarchical saliency map S obtained in the T-th layer learning process is shown.
Let S be the final saliency map SfinalIn any significant area, the position of the airport target or the oil tank target is marked in the input image to be detected according to the label of the area, namely the airport target or the oil tank target, determined by the self-adaptive learning ending criterion, so that the task of detecting the remote sensing target is completed.
The invention has the following advantages and beneficial effects:
(1) the invention adopts the similarity measurement operator with the minimum distance to carry out the similarity measurement of the feature vector, and a similarity measurement coefficient can be obtained corresponding to each superpixel and is used as the potential target feature of the superpixel, thereby learning the feature difference between each superpixel and the background information set.
(2) The invention adopts a hierarchical enhanced structural framework to adaptively learn the potential target characteristics. During each layer of learning and updating process of the image, the feature representation of the target area is made more prominent, and simultaneously the feature representation of the background area is suppressed, so that the prominent target is gradually highlighted.
(3) The invention adaptively determines the number of learning layers. When the target area in the hierarchical saliency map is sufficiently salient, the model can automatically control the learning process to end, so that the number of learned layers is adaptively determined, and manual intervention is reduced, so that the algorithm has good adaptability to different input images.
(4) The invention adopts a super-pixel segmentation method from fine to coarse. When superpixel segmentation is carried out in hierarchical learning, fine segmentation is carried out first, and then coarse segmentation is carried out. The method comprises the steps of firstly carrying out fine segmentation, improving the accuracy of segmentation, facilitating the acquisition of accurate boundary information of a target object in an image, and then carrying out coarse segmentation, and improving the rapidity of an algorithm on the basis of ensuring the accuracy.
(5) When a hierarchical reinforcement learning model is improved, a mode of fusing a top-down target feature map and a bottom-up underlying potential target feature map is provided, so that candidate significant regions with relatively significant colors and textures can be extracted from a large-amplitude remote sensing image, significant regions meeting the requirement of a detection target task can be screened out from the candidate significant regions according to task driving of target detection, and the constructed significance detection model has pertinence and flexibility.
(6) The invention applies the LDA topic model as the judgment condition of learning ending, and the LDA topic model can extract the topic characteristics of any target, so the method can process the detection problem of multiple targets, and the method can determine the target category while determining the significant area.
The improved remote sensing target detection method of the hierarchical significant model can accurately detect the airport and oil tank targets in the low-resolution remote sensing images under different sizes and illumination conditions, and has better robustness.
Drawings
FIG. 1 is a detailed flowchart of a remote sensing target detection method for an improved hierarchical saliency model according to the present invention;
FIG. 2 is a diagram of airport detection effects of the remote sensing target detection algorithm under various scales and illumination conditions in the present invention; wherein a is a detection effect diagram of a small-size airport target, b is a detection effect diagram of a large-size airport target, and c is a detection effect diagram of an airport target under the condition of insufficient illumination;
fig. 3 is a schematic diagram of the detection result of the remote sensing target detection algorithm for the airport and oil tank mixed target, wherein a is the detection result of the airport target, and b is the detection result of the oil tank target.
Detailed Description
Referring to fig. 1, a remote sensing target detection method based on an improved hierarchical significant model according to the present invention is described in an embodiment by taking an airport and oil tank mixed target as an example, and includes the following specific implementation steps:
step 1: performing superpixel segmentation on an input image to be detected by using a simple linear iterative clustering algorithm SLIC (linear iterative clustering algorithm), clustering pixels with color similarity in a neighboring region in the image, and representing the regions clustered into a class by adopting superpixels to obtain an image subjected to superpixel segmentation;
the method comprises the steps of taking color features of a color image and position information of each pixel point as constraints for an input image I to be detected, clustering by adopting a K-means clustering algorithm, extracting color features of a CIELab color space of the image, representing local pixel points with similar color features by using superpixels, completing a superpixel segmentation process of the image, selecting the total superpixel segmentation number of the whole image as K, and enabling the segmented image I to comprise K superpixel regions.
Step 2: taking the color feature of each superpixel in the segmented image as a bottom layer feature, constructing a global information set, and simultaneously extracting the bottom layer features of all superpixels at the boundary position of the segmented image to construct a background information set;
(1) knowing that the total number of superpixels contained in the image after superpixel segmentation is k, selecting the color feature of each superpixel to construct the bottom layer feature, namely converting the image into CIELab color space, respectively solving the values of all pixels in each superpixel in three color channels of the CIELab color space, and taking the mean value of all pixel points in each superpixel in each color channel as the bottom layer feature of the superpixel, namely the bottom layer feature p of the ith superpixeliExpressed as:
pi=(lli,lai,lbi) (1)
wherein, lli,lai,lbiRespectively representing the average value of all pixel points in the ith super pixel in the CIELab color space in the values of the three color channels, and then the feature set of all super pixels is P ═ { P ═1,p2,…,pkDefining P as a global information set; wherein p is1,p2,…,pkRespectively representing the bottom layer characteristics of 1 st, 2 nd, … th k super pixel areas in the segmented image; i is more than or equal to 1 and less than or equal to k, i represents any super pixel in the global information set, and k represents the total number of the super pixels;
(2) for a global information set P ═ P1,p2,…,pkN super-pixels are arranged on the image boundary, the n super-pixels on the boundary are extracted to form a background information set, n represents the total number of the background super-pixels, and a background information set B represents:
B={b1,b2,…,bn},0<n<k (2)
bj=(llj,laj,lbj),1≤j≤n (3)
where j represents any super-pixel in the background information set, bjUnderlying features, ll, representing super-pixel regions in a background information setj,laj,lbjRespectively representing the average values of all pixel points in the jth super pixel of the CIELab color space in the values of the three color channels.
And step 3: learning the feature similarity between each super-pixel and a background information set by adopting a similarity measurement operator LDSM based on a minimum distance, wherein the feature similarity is represented by a learning coefficient of the LDSM, the value of the learning coefficient is in positive correlation with the feature similarity between each super-pixel and the background information set, and a potential target feature map is constructed by utilizing the learning coefficient, can reflect the feature difference between each super-pixel region and the background information set and comprises all candidate significant target regions;
defining a superpixel dataset (p)i,bj) Wherein p isiBottom layer features representing super-pixel regions in the global feature set, bjRepresenting an underlying feature of a super-pixel region in a background information set, i.e. pi∈P,bj∈ B, the LDSM operator is used to obtain the bottom characteristic p of the super-pixel area in each global information setiWith the underlying features b of the super-pixel region in each background information setjCoefficient of similarity between αijAs shown in the following formula:
by solving equation (4), the learning coefficient α of the similarity measure is obtainedijWhen p isiAnd bjWhen they are close, αijIs approximately 1; when p isiAnd bjWhen they are completely equal αijEqual to 1; when p isiAnd bjWhen the difference is large, αijAway from 1; due to piAnd bjIs expressed as αijDegree of deviation from 1, the similarity learning coefficient αijNormalized to β according to equation (5)ijThen, then
Wherein,represents will | aij-1| is normalized to the interval [0,1 |)]Normalized learning coefficient βijThe closer to 0, the lower level feature p representing the global superpixel regioniBottom layer feature b closer to background super pixel areaj;
By solving the above optimization problem, a set of normalized learning coefficients is obtained, as shown in the following equation:
i represents any row of the matrix in equation (6), the ith row represents the learning coefficient obtained by learning all background superpixels by the ith superpixel in the feature set, the potential features β are defined by taking the mean value of each element in each rowiThe calculation formula of (2) is as follows:
where n is the total background superpixel number, βijRepresenting any one element in the matrix of the formula (6), and obtaining a potential target feature map F (β) from the formula (7)1,β2,…,βk) And k represents the total number of superpixels.
And 4, step 4: extracting prior characteristics of airport and oil tank targets from top to bottom from an input image to be detected, and respectively generating target characteristic graphs. For the airport target, performing linear detection on an input image to be detected by using an LSD (Line Segment Detector) to obtain a linear detection result, then counting the number of pixel points on the detected linear in each super-pixel region by using the image obtained by super-pixel segmentation in the step 1, and generating a linear density graph as a target feature graph of the airport target; for an oil tank target, carrying out circle detection on an input image to be detected by using a Hough transformation method to obtain a circle detection result, distributing different weights to the detected circle internal points and non-internal points by using a voting mechanism to generate a circle feature map, and taking the circle feature map as a target feature map of the oil tank target;
(1) extracting airport target characteristic diagram
For an input image I to be detected, initially acquiring straight line information of the whole image by applying an LSD (least squares) straight line detection operator, then calculating straight line density characteristics corresponding to a super-pixel segmentation result of a current layer, wherein a numerator is the number of pixel points on a straight line in each super-pixel region, and a denominator is the area of a corresponding region. The corresponding linear density characteristic d of any super pixel region i is expressed by the formula (8)iComprises the following steps:
wherein NumLRepresenting the number of pixels on a straight line, Num being the total number of pixels, region (i) representing the ith super-pixelThe region is obtained from the region, namely a linear density map D ═ D (1,d2,…,dk) Wherein d is1,d2,…,dkRespectively representing the linear density characteristics of 1, 2, … th superpixel areas, wherein k represents the total number of superpixels;
(2) extracting oil tank target characteristic diagram
The circle feature is a typical regional feature of an oil tank target, circle detection is carried out on an input image to be detected by utilizing Hough transformation, then a circle detection result is converted into a feature diagram, and the feature diagram is used as an oil tank target feature diagram;
when circle detection is carried out by utilizing Hough transformation, voting is carried out on (a, c, r) consisting of circle center positions (a, c) and circle radius r, wherein a is a circle center abscissa and c is a circle center ordinate; the local peak value in the voting result is the existing center position coordinate and radius of the circle, the voting result is used as the weight value of the circle feature, and all points meeting the formula (9) should share the same weight value as the corresponding (a, c, r), that is, all points should share the same weight value as the corresponding (a, c, r), that is, the local peak value in the voting result is the existing center position coordinate and radius of the circle
(x-a)2+(y-c)2≤r2(9)
Wherein (x, y) is the coordinate of any position in the target characteristic diagram, and the original weight values of other points in the non-circle are retained to obtain a target characteristic diagram C of the oil tank target.
And 5: fusing the potential target characteristic diagram F in the step 3 with the airport target characteristic diagram D and the oil tank target characteristic diagram C in the step 4 to generate a hierarchical saliency map; the salient regions in the hierarchical salient map include candidate salient target regions, airport target regions and tank target regions in the potential target feature map obtained at each layer in the learning process.
Firstly, carrying out additive fusion on a target characteristic diagram D of an airport target and a target characteristic diagram C of an oil tank target, namely performing union operation to obtain an image subjected to additive fusion; and then fusing the potential target feature graph F and the image after the addition fusion again, wherein the fusion process adopts a mode of multiplication among corresponding pixel points, namely intersection operation is taken, and the fusion mode is expressed as follows:
S=(D+C)·F (10)
wherein, S represents a saliency map generated through fusion in each layer learning process, i.e. a hierarchical saliency map.
Step 6: and defining an adaptive learning ending criterion by using the LDA topic model, and judging the feature similarity between the salient region in the current-level saliency map and the airport and tank targets in the step 5 by using the criterion so as to judge whether the current learning process should be ended. If the learning process is not finished, executing step 7; if the learning process has ended, go to step 8;
(1) firstly, obtaining sample images of an airport target and an oil tank target from a database, respectively training an LDA model by using the sample of the airport target and the sample of the oil tank target to obtain a theme model of the airport target and the oil tank target, and simultaneously training a theme model of a background to obtain the theme model of the airport target, the oil tank target and the background. In the LDA model training process, the characteristics of all training samples are selected as the color characteristics of a CIELab color space; after LDA model training, a background topic model p (z | bg) and an airport topic model p (z | fg) are obtained1) And tank topic model p (z | fg)2);
(2) For the level saliency map of each layer, a threshold of 0.6 is set. When the value of the super pixel area is more than 0.6, the super pixel area is considered as a significant area, otherwise, the super pixel area is considered as not significant; computing a topic model p (zs) for the ith salient regioni) Wherein s isiRepresenting the ith super-pixel salient region, a topic model p (z | s) is calculatedi) P (z | bg), p (z | fg)1),p(z|fg2) The distance between the three types of the three is taken as the type label corresponding to the topic model with the minimum distance, wherein the type label is one of an airport target, an oil tank target or a background, and the three are given to a significant area siThe same label is labeled, and when the type labels of all the salient areas are the type labels of the targets,finishing learning, and outputting the position and the type label value of each salient region, otherwise, continuing the reinforcement learning process of the next layer; the cosine distance between the two vectors is used when calculating the distance between the topic models.
And 7: and a multi-layer learning framework is adopted, and a feedback mechanism is utilized between adjacent layers to feed the level saliency map obtained by each layer back to the input image of the current layer, so that the background suppression layer by layer is realized. Namely, the hierarchical saliency map obtained through fusion in the step 5 is used as an enhancement factor to act on the input image of the current layer, the image of the current layer after superpixel segmentation is enhanced, a new image obtained through enhancement is used as the input image for next-layer learning, the step 1 is executed, and the learning process of a new layer is started; repeating the steps, and gradually highlighting the target area through multi-layer learning;
when the learned hierarchical saliency map S of the current layer does not satisfy the learning termination condition, the hierarchical saliency map S is first stretched by a stretching function, which is as follows:
R=f(S) (11)
wherein f represents a stretching function applied to the hierarchical saliency map S, said stretching function selecting a quadratic function; and defining the stretched matrix R as an enhancement matrix, and applying the enhancement matrix to enhance the input image. The image enhancement formula for the first layer is as follows:
I2=I·R1(12)
R1is an enhanced matrix of a first layer in the learning process, and an enhanced image I of the first layer is obtained by the feedback between layers of an input image I to be detected2Is shown by2As input image for the second layer reinforcement learning process, pair I2And (3) performing the characteristic extraction step to obtain a level saliency map of the second-level learning process, and by analogy, the feedback expression among levels is as follows:
It+1=It·Rt(13)
wherein, ItAnd It+1Representing input images of the t-th and t + 1-th layers, respectively, RtFor the enhancement matrix of the t-th layer, the input image I of the t + 1-th layer is processed by the inter-level feedback process of formula (13)t+1The saliency of the target area is preserved while the background area is suppressed;
in the t-th learning process, the input image I of the current layertPerforming superpixel segmentation on the image by using SLIC algorithm, and segmenting the image into ktA super pixel region; the adopted super-pixel segmentation number meets the following conditions:
k1≥k2≥k3≥…≥kt≥…,t=1,2,3,… (14)
namely, a segmentation mode from fine to coarse is adopted, the edge characteristics of the image can be accurately reserved by the initial fine segmentation, and the calculation amount can be properly reduced by the subsequent coarse segmentation;
updated image It+1The learning of a new layer is started with the input image of the next layer, and step 1 is executed.
And 8: and after the learning is stopped, taking the level saliency map obtained in the step 5 in the current layer as a final saliency map, extracting a saliency region in the final saliency map as a target region, and marking a target region and a category label of the target, namely an airport target or an oil tank target, in the image to be detected, so as to obtain a final result of remote sensing target detection.
In the learning process, if the level saliency map of the current layer meets the self-adaptive learning ending criterion, ending the learning process; the number T of layers where the current layer is located is represented as the total number of learning layers and is also the iteration number of the learning process; taking the hierarchical saliency map of the current layer as a final saliency map, namely a final saliency map SfinalExpressed as:
Sfinal=ST(15)
STrepresenting layers obtained during the T-th layer learning processLevel saliency map S.
Let S be the final saliency map SfinalIn any significant area, the position of the airport target or the oil tank target is marked in the input image to be detected according to the label of the area, namely the airport target or the oil tank target, determined by the self-adaptive learning ending criterion, so that the task of detecting the remote sensing target is completed.
FIG. 2 is a diagram of the airport detection effect of the remote sensing target detection algorithm under various scales and illumination conditions, and (a) is a diagram of the detection effect of small-size airports. The detection result shows that the remote sensing target detection method provided by the invention can better detect the whole airport area, can well suppress the interference of the surrounding background area, and can detect the obvious airport area completely. Wherein (b) is a detection effect graph for large-size airport targets. As can be seen from the detection results, under the condition of large scale, the detection results of the airport targets are complete. (c) The method is a detection effect diagram for the airport target under the condition of insufficient illumination. As can be seen from the detection result, when the illumination is insufficient, the target detection model can still accurately extract the position of the airport target. As can be seen from FIG. 2, the target detection model has good self-adaptive capacity for the earth surface environment of the remote sensing image, the target dimension and the illumination change, namely the algorithm has the characteristics of wide application range and good robustness.
FIG. 3 is a schematic diagram of the detection result of the remote sensing target detection algorithm for the airport and oil tank mixed target. (a) The method can accurately detect the target area, has good edge retentivity to the remote sensing target, and can highlight the whole target area at the same time. (b) Is the detection result of the oil tank object, and the clutter degree of the background is slightly increased from the detection result, because a plurality of oil tank objects usually exist in one image, and the pressing of the background is completely performed when a single object is not pressed. But despite the slightly cluttered background, the most salient areas in the map can still accurately hit the target.
Claims (9)
1. A remote sensing target detection method based on an improved hierarchical significant model is characterized by comprising the following steps: the remote sensing target mainly aims at two types of targets, namely an airport target and an oil tank target, and comprises the following steps:
step 1: performing superpixel segmentation on an input image to be detected by using a Simple linear iterative Clustering algorithm (SLIC), Clustering pixels with color similarity in a neighboring region in the image, and representing the regions clustered into a class by using superpixels so as to obtain an image subjected to superpixel segmentation;
step 2: taking the color feature of each superpixel in the segmented image as a bottom layer feature, constructing a global information set, and simultaneously extracting the bottom layer features of all superpixels positioned at the boundary position of the segmented image to construct a background information set;
and step 3: learning the feature similarity between each super pixel and a background information set by adopting a minimum Distance-based similarity measurement operator LDSM (LDSM), wherein the feature similarity is represented by a learning coefficient of the LDSM, the value of the learning coefficient is in positive correlation with the feature similarity between each super pixel and the background information set, and a potential target feature map is constructed by utilizing the learning coefficient and can reflect the feature difference between each super pixel region and the background information set and comprises all candidate significant target regions;
and 4, step 4: extracting prior characteristics of an airport target and an oil tank target from top to bottom from an input image to be detected, and respectively generating target characteristic graphs; for an airport target, performing linear detection on an input image to be detected by using a Linear Segment Detector (LSD) to obtain a linear detection result; then, counting the number of pixel points on the detected straight line in each super pixel region by using the image obtained by super pixel segmentation in the step 1, and generating a straight line density graph as a target characteristic graph of the airport target; for an oil tank target, carrying out circle detection on an input image to be detected by using a Hough transformation method to obtain a circle detection result, distributing different weights to the detected circle internal points and non-internal points by using a voting mechanism to generate a circle feature map, and taking the circle feature map as a target feature map of the oil tank target;
and 5: fusing the potential target feature map in the step 3 with the airport target feature map and the oil tank target feature map in the step 4, namely three images, and generating a level saliency map; the salient regions in the hierarchical salient map comprise candidate salient target regions, airport target regions and oil tank target regions in the potential target feature map obtained by each layer in the learning process;
step 6: defining an adaptive learning ending criterion by using a Latent topic semantic model LDA (LDA), judging the feature similarity between the salient region in the current-level salient map in the step 5 and the airport and oil tank target by using the criterion, and judging whether the current learning process should be ended or not, if the learning process is not ended, executing a step 7; if the learning process has ended, go to step 8;
and 7: feeding the level saliency map obtained from each layer back to the input image of the current layer by adopting a multilayer learning frame and utilizing a feedback mechanism between adjacent layers to realize background suppression layer by layer, namely taking the level saliency map obtained through fusion in the step 5 as an enhancement factor to act on the input image of the current layer, enhancing the image of the current layer after superpixel segmentation, taking a new image obtained after enhancement as the input image of the next layer for learning, executing the step 1 and starting the learning process of a new layer; repeating the steps, and gradually highlighting the target area through multi-layer learning;
and 8: and after the learning is stopped, taking the level saliency map obtained in the step 5 in the current layer as a final saliency map, extracting a saliency region in the final saliency map as a target region, and marking the target region and a category label of the target, namely an airport target or an oil tank target, in the image to be detected, so as to obtain a final result of remote sensing target detection.
2. The remote sensing target detection method based on the improved hierarchical significant model according to claim 1, characterized in that: in the step 1, the specific steps of performing super-pixel segmentation by using the SLIC algorithm are as follows:
the method comprises the steps of taking color features of a color image and position information of each pixel point as constraints for an input image I to be detected, clustering by adopting a K-means clustering algorithm, extracting color features of a CIELab color space of the image, representing local pixel points with similar color features by using superpixels, completing a superpixel segmentation process of the image, selecting the total superpixel segmentation number of the whole image as K, and enabling the segmented image I to comprise K superpixel regions.
3. The remote sensing target detection method based on the improved hierarchical significant model according to claim 1, characterized in that: in step 2, the specific steps of constructing the global information set and the background information set are as follows:
(1) knowing that the total number of superpixels contained in the image after superpixel segmentation is k, selecting the color feature of each superpixel to construct the bottom layer feature, namely converting the image into CIELab color space, respectively solving the values of all pixels in each superpixel in three color channels of the CIELab color space, and taking the mean value of all pixel points in each superpixel in each color channel as the bottom layer feature of the superpixel, namely the bottom layer feature p of the ith superpixeliExpressed as:
pi=(lli,lai,lbi) (1)
wherein, lli,lai,lbiRespectively representing the average value of all pixel points in the ith super pixel in the CIELab color space in the values of the three color channels, and then the feature set of all super pixels is P ═ { P ═1,p2,…,pkDefining P as a global information set; wherein p is1,p2,…,pkRespectively representing the bottom layer characteristics of 1 st, 2 nd, … th k super pixel areas in the segmented image; i is more than or equal to 1 and less than or equal to k, i represents any super pixel in the global information set, and k represents the total number of the super pixels;
(2) for a global information set P ═ P1,p2,…,pkN super-pixels are arranged on the image boundary, the n super-pixels on the boundary are extracted to form a background information set, n represents the total number of the background super-pixels, and a background information set B represents:
B={b1,b2,…,bn},0<n<k (2)
bj=(llj,laj,lbj),1≤j≤n (3)
where j represents any super-pixel in the background information set, bjUnderlying features, ll, representing super-pixel regions in a background information setj,laj,lbjRespectively representing the average values of all pixel points in the jth super pixel of the CIELab color space in the values of the three color channels.
4. The remote sensing target detection method based on the improved hierarchical significant model according to claim 1, characterized in that: in the step 3, a potential target feature map is constructed by using the learning coefficient, which specifically includes the following steps:
defining a superpixel dataset (p)i,bj) Wherein p isiBottom layer features representing super-pixel regions in the global feature set, bjRepresenting an underlying feature of a super-pixel region in a background information set, i.e. pi∈P,bj∈ B, the LDSM operator is used to obtain the bottom characteristic p of the super-pixel area in each global information setiWith the underlying features b of the super-pixel region in each background information setjCoefficient of similarity between αijAs shown in the following formula:
by solving equation (4), the learning coefficient α of the similarity measure is obtainedijWhen p isiAnd bjWhen they are close, αijIs approximately 1; when p isiAnd bjWhen they are completely equal αijEqual to 1; when p isiAnd bjWhen the difference is large, αijAway from 1; due to piAnd bjIs expressed as αijDegree of deviation from 1, the similarity learning coefficient αijNormalized to β according to equation (5)ijThen, then
Wherein,represents will | aij-1| is normalized to the interval [0,1 |)]Normalized learning coefficient βijThe closer to 0, the lower level feature p representing the global superpixel regioniBottom layer feature b closer to background super pixel areaj;
By solving the above optimization problem, a set of normalized learning coefficients is obtained, as shown in the following equation:
i represents any row of the matrix in equation (6), the ith row represents the learning coefficient obtained by learning all background superpixels by the ith superpixel in the feature set, the potential features β are defined by taking the mean value of each element in each rowiThe calculation formula of (2) is as follows:
where n is the total background superpixel number, βijRepresenting any one element in the matrix of the formula (6), and obtaining a potential target feature map F (β) from the formula (7)1,β2,…,βk) And k represents the total number of superpixels.
5. The remote sensing target detection method based on the improved hierarchical significant model according to claim 1, characterized in that: in the step 4, the target feature maps of the airport and oil tank targets are extracted, and the following steps are realized:
(1) extracting airport target characteristic diagram
For an input image I to be detected, initially acquiring straight line information of the whole image by applying an LSD (least squares) straight line detection operator, then calculating straight line density characteristics corresponding to a super-pixel segmentation result of a current layer, wherein a numerator is the number of pixel points on a straight line in each super-pixel region, and a denominator is the area of a corresponding region. The corresponding linear density characteristic d of any super pixel region i is expressed by the formula (8)iComprises the following steps:
wherein NumLRepresenting the number of pixel points on a straight line, Num is the total number of pixels, region (i) represents the ith super pixel region, and therefore, an object feature map based on the airport object is obtained, namely a straight line density map D ═ D1,d2,…,dk) Wherein d is1,d2,…,dkRespectively representing the linear density characteristics of 1, 2, … th superpixel areas, wherein k represents the total number of superpixels;
(2) extracting oil tank target characteristic diagram
The circle feature is a typical regional feature of an oil tank target, circle detection is carried out on an input image to be detected by utilizing Hough transformation, then a circle detection result is converted into a feature diagram, and the feature diagram is used as an oil tank target feature diagram;
when circle detection is carried out by utilizing Hough transformation, voting is carried out on (a, c, r) consisting of circle center positions (a, c) and circle radius r, wherein a is a circle center abscissa and c is a circle center ordinate; the local peak in the voting result is the existing coordinates of the center of circle and the radius of the circle, the voting result is used as the weight value of the circle feature, and all points satisfying the formula (9) should share the same weight value as the corresponding (a, c, r), that is:
(x-a)2+(y-c)2≤r2(9)
wherein (x, y) is the coordinate of any position in the target characteristic diagram, and the original weight values of other points in the non-circle are retained to obtain a target characteristic diagram C of the oil tank target.
6. The remote sensing target detection method based on the improved hierarchical significant model according to claim 1, characterized in that: in the step 5, the fusion generation of the hierarchical saliency map is specifically as follows:
firstly, carrying out additive fusion on a target characteristic diagram D of an airport target and a target characteristic diagram C of an oil tank target, namely performing union operation to obtain an image subjected to additive fusion; and then fusing the potential target feature graph F and the image after the addition fusion again, wherein the fusion process adopts a mode of multiplication among corresponding pixel points, namely intersection operation is taken, and the fusion mode is expressed as follows:
S=(D+C)·F (10)
wherein, S represents a saliency map generated through fusion in each layer learning process, i.e. a hierarchical saliency map.
7. The remote sensing target detection method based on the improved hierarchical significant model according to claim 1, characterized in that: in step 6, defining an adaptive learning termination criterion, and determining whether to terminate the current learning process, specifically including:
(1) firstly, obtaining sample images of an airport target and an oil tank target from a database, respectively training an LDA model by using the sample of the airport target and the sample of the oil tank target to obtain a theme model of the airport target and the oil tank target, and simultaneously training a theme model of a background to obtain the theme model of the airport target, the oil tank target and the background. In the LDA model training process, the characteristics of all training samples are selected as the color characteristics of a CIELab color space; after LDA model training, a background topic model p (z | bg) and an airport topic model p (z | fg) are obtained1) And tank topic model p (z | fg)2);
(2) For the level saliency map of each layer, a threshold of 0.6 is set. When the value of the super pixel area is more than 0.6, the super pixel area is considered as a significant area, otherwise, the super pixel area is considered as not significant; computing a topic model p (zs) for the ith salient regioni) Wherein s isiRepresenting the ith super-pixel salient region, a topic model p (z | s) is calculatedi) P (z | bg), p (z | fg)1),p(z|fg2) The type label corresponding to the topic model with the minimum distance is taken, the type label is one of an airport target, an oil tank target or a background,giving a salient region siLabeling the same labels, finishing learning when the type labels of all the salient regions are the type labels of the targets, and outputting the position of each salient region and the type label value of the salient region, otherwise, continuing the reinforcement learning process of the next layer; the cosine distance between the two vectors is used when calculating the distance between the topic models.
8. The remote sensing multi-target detection method based on the improved hierarchical significant model according to claim 1, characterized in that: in the step 7, a multi-layer learning framework is adopted, and a feedback mechanism is utilized between adjacent layers to feed back the level saliency map of each layer to the input image of the current layer, so as to realize background suppression layer by layer, specifically comprising:
when the learned hierarchical saliency map S of the current layer does not satisfy the learning termination condition, the hierarchical saliency map S is first stretched by a stretching function, which is as follows:
R=f(S) (11)
wherein f represents a stretching function applied to the hierarchical saliency map S, said stretching function selecting a quadratic function; and defining the stretched matrix R as an enhancement matrix, and applying the enhancement matrix to enhance the input image. The image enhancement formula for the first layer is as follows:
I2=I·R1(12)
R1is an enhanced matrix of a first layer in the learning process, and an enhanced image I of the first layer is obtained by the feedback between layers of an input image I to be detected2Is shown by2As input image for the second layer reinforcement learning process, pair I2And (3) performing the characteristic extraction step to obtain a level saliency map of the second-level learning process, and by analogy, the feedback expression among levels is as follows:
It+1=It·Rt(13)
wherein, ItAnd It+1Representing input images of the t-th and t + 1-th layers, respectively, RtFor the enhancement matrix of the t-th layer, the input image I of the t + 1-th layer is processed by the inter-level feedback process of formula (13)t+1Target areaWhile the background area is suppressed while the saliency of (1) is preserved;
in the t-th learning process, the input image I of the current layertPerforming superpixel segmentation on the image by using SLIC algorithm, and segmenting the image into ktA super pixel region; the adopted super-pixel segmentation number meets the following conditions:
k1≥k2≥k3≥…≥kt≥…,t=1,2,3,… (14)
namely, a segmentation mode from fine to coarse is adopted, the edge characteristics of the image can be accurately reserved by the initial fine segmentation, and the calculation amount can be properly reduced by the subsequent coarse segmentation;
updated image It+1The learning of a new layer is started with the input image of the next layer, and step 1 is executed.
9. The remote sensing target detection method based on the improved hierarchical significant model according to claim 1, characterized in that: in step 8, obtaining the target area and the category label of the target in the final saliency map specifically includes:
in the learning process, if the level saliency map of the current layer meets the self-adaptive learning ending criterion, ending the learning process; the number T of layers where the current layer is located is represented as the total number of learning layers and is also the iteration number of the learning process; taking the hierarchical saliency map of the current layer as a final saliency map, namely a final saliency map SfinalExpressed as:
Sfinal=ST(15)
STa hierarchy saliency map S obtained in the T-th layer learning process is represented;
let S be the final saliency map SfinalIn any significant area, the position of the airport target or the oil tank target is marked in the input image to be detected according to the label of the area, namely the airport target or the oil tank target, determined by the self-adaptive learning ending criterion, so that the task of detecting the remote sensing target is completed.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710115840.6A CN106909902B (en) | 2017-03-01 | 2017-03-01 | Remote sensing target detection method based on improved hierarchical significant model |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710115840.6A CN106909902B (en) | 2017-03-01 | 2017-03-01 | Remote sensing target detection method based on improved hierarchical significant model |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106909902A true CN106909902A (en) | 2017-06-30 |
CN106909902B CN106909902B (en) | 2020-06-05 |
Family
ID=59209087
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710115840.6A Active CN106909902B (en) | 2017-03-01 | 2017-03-01 | Remote sensing target detection method based on improved hierarchical significant model |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106909902B (en) |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107527023A (en) * | 2017-08-07 | 2017-12-29 | 西安理工大学 | Classification of Polarimetric SAR Image method based on super-pixel and topic model |
CN107633491A (en) * | 2017-09-26 | 2018-01-26 | 中国科学院长春光学精密机械与物理研究所 | A kind of area image Enhancement Method and storage medium based on target detection |
CN107688782A (en) * | 2017-08-23 | 2018-02-13 | 中国科学院软件研究所 | Oil tank detection and reserve analysis method based on high-resolution optical remote sensing image |
CN108596055A (en) * | 2018-04-10 | 2018-09-28 | 西北工业大学 | The airport target detection method of High spatial resolution remote sensing under a kind of complex background |
CN108629286A (en) * | 2018-04-03 | 2018-10-09 | 北京航空航天大学 | A kind of remote sensing airport target detection method based on the notable model of subjective perception |
CN109033998A (en) * | 2018-07-04 | 2018-12-18 | 北京航空航天大学 | Remote sensing image atural object mask method based on attention mechanism convolutional neural networks |
CN109543561A (en) * | 2018-10-31 | 2019-03-29 | 北京航空航天大学 | Saliency of taking photo by plane method for detecting area and device |
CN109726649A (en) * | 2018-12-15 | 2019-05-07 | 中国科学院深圳先进技术研究院 | Remote sensing image cloud detection method of optic, system and electronic equipment |
CN110047076A (en) * | 2019-03-29 | 2019-07-23 | 腾讯科技(深圳)有限公司 | A kind of processing method of image information, device and storage medium |
CN110097569A (en) * | 2019-04-04 | 2019-08-06 | 北京航空航天大学 | Oil tank object detection method based on color Markov Chain conspicuousness model |
CN110458192A (en) * | 2019-07-05 | 2019-11-15 | 中国地质大学(武汉) | The Classification of hyperspectral remote sensing image method and system of view-based access control model conspicuousness |
CN111209918A (en) * | 2020-01-06 | 2020-05-29 | 河北工业大学 | Image saliency target detection method |
CN112862765A (en) * | 2021-01-26 | 2021-05-28 | 重庆师范大学 | Soil color image shadow detection method based on semi-supervised dispersion |
CN116630820A (en) * | 2023-05-11 | 2023-08-22 | 北京卫星信息工程研究所 | Optical remote sensing data on-satellite parallel processing method and device |
CN117808703A (en) * | 2024-02-29 | 2024-04-02 | 南京航空航天大学 | Multi-scale large-scale component assembly gap point cloud filtering method |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102831402A (en) * | 2012-08-09 | 2012-12-19 | 西北工业大学 | Sparse coding and visual saliency-based method for detecting airport through infrared remote sensing image |
CN105930868A (en) * | 2016-04-20 | 2016-09-07 | 北京航空航天大学 | Low-resolution airport target detection method based on hierarchical reinforcement learning |
-
2017
- 2017-03-01 CN CN201710115840.6A patent/CN106909902B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102831402A (en) * | 2012-08-09 | 2012-12-19 | 西北工业大学 | Sparse coding and visual saliency-based method for detecting airport through infrared remote sensing image |
CN105930868A (en) * | 2016-04-20 | 2016-09-07 | 北京航空航天大学 | Low-resolution airport target detection method based on hierarchical reinforcement learning |
Non-Patent Citations (1)
Title |
---|
DANPEI ZHAO等: "Hierarchical reinforcement learning for saliency detection of low-resolution airports", 《2016 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM (IGARSS)》 * |
Cited By (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107527023A (en) * | 2017-08-07 | 2017-12-29 | 西安理工大学 | Classification of Polarimetric SAR Image method based on super-pixel and topic model |
CN107527023B (en) * | 2017-08-07 | 2021-05-25 | 西安理工大学 | Polarized SAR image classification method based on superpixels and topic models |
CN107688782A (en) * | 2017-08-23 | 2018-02-13 | 中国科学院软件研究所 | Oil tank detection and reserve analysis method based on high-resolution optical remote sensing image |
CN107633491A (en) * | 2017-09-26 | 2018-01-26 | 中国科学院长春光学精密机械与物理研究所 | A kind of area image Enhancement Method and storage medium based on target detection |
CN108629286A (en) * | 2018-04-03 | 2018-10-09 | 北京航空航天大学 | A kind of remote sensing airport target detection method based on the notable model of subjective perception |
CN108629286B (en) * | 2018-04-03 | 2021-09-28 | 北京航空航天大学 | Remote sensing airport target detection method based on subjective perception significance model |
CN108596055A (en) * | 2018-04-10 | 2018-09-28 | 西北工业大学 | The airport target detection method of High spatial resolution remote sensing under a kind of complex background |
CN108596055B (en) * | 2018-04-10 | 2022-02-11 | 西北工业大学 | Airport target detection method of high-resolution remote sensing image under complex background |
CN109033998A (en) * | 2018-07-04 | 2018-12-18 | 北京航空航天大学 | Remote sensing image atural object mask method based on attention mechanism convolutional neural networks |
CN109543561A (en) * | 2018-10-31 | 2019-03-29 | 北京航空航天大学 | Saliency of taking photo by plane method for detecting area and device |
CN109726649A (en) * | 2018-12-15 | 2019-05-07 | 中国科学院深圳先进技术研究院 | Remote sensing image cloud detection method of optic, system and electronic equipment |
CN110047076B (en) * | 2019-03-29 | 2021-03-23 | 腾讯科技(深圳)有限公司 | Image information processing method and device and storage medium |
CN110047076A (en) * | 2019-03-29 | 2019-07-23 | 腾讯科技(深圳)有限公司 | A kind of processing method of image information, device and storage medium |
CN110097569A (en) * | 2019-04-04 | 2019-08-06 | 北京航空航天大学 | Oil tank object detection method based on color Markov Chain conspicuousness model |
CN110458192A (en) * | 2019-07-05 | 2019-11-15 | 中国地质大学(武汉) | The Classification of hyperspectral remote sensing image method and system of view-based access control model conspicuousness |
CN110458192B (en) * | 2019-07-05 | 2022-06-14 | 中国地质大学(武汉) | Hyperspectral remote sensing image classification method and system based on visual saliency |
CN111209918A (en) * | 2020-01-06 | 2020-05-29 | 河北工业大学 | Image saliency target detection method |
CN111209918B (en) * | 2020-01-06 | 2022-04-05 | 河北工业大学 | Image saliency target detection method |
CN112862765A (en) * | 2021-01-26 | 2021-05-28 | 重庆师范大学 | Soil color image shadow detection method based on semi-supervised dispersion |
CN112862765B (en) * | 2021-01-26 | 2022-08-02 | 重庆师范大学 | Soil color image shadow detection method based on semi-supervised dispersion |
CN116630820A (en) * | 2023-05-11 | 2023-08-22 | 北京卫星信息工程研究所 | Optical remote sensing data on-satellite parallel processing method and device |
CN116630820B (en) * | 2023-05-11 | 2024-02-06 | 北京卫星信息工程研究所 | Optical remote sensing data on-satellite parallel processing method and device |
CN117808703A (en) * | 2024-02-29 | 2024-04-02 | 南京航空航天大学 | Multi-scale large-scale component assembly gap point cloud filtering method |
CN117808703B (en) * | 2024-02-29 | 2024-05-10 | 南京航空航天大学 | Multi-scale large-scale component assembly gap point cloud filtering method |
Also Published As
Publication number | Publication date |
---|---|
CN106909902B (en) | 2020-06-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106909902B (en) | Remote sensing target detection method based on improved hierarchical significant model | |
CN105930868B (en) | A kind of low resolution airport target detection method based on stratification enhancing study | |
CN108573276B (en) | Change detection method based on high-resolution remote sensing image | |
CN104915636B (en) | Remote sensing image road recognition methods based on multistage frame significant characteristics | |
US10217236B2 (en) | Remote determination of containers in geographical region | |
Chen et al. | Vehicle detection in high-resolution aerial images via sparse representation and superpixels | |
CN103049763B (en) | Context-constraint-based target identification method | |
CN108776779B (en) | Convolutional-circulation-network-based SAR sequence image target identification method | |
CN108596055B (en) | Airport target detection method of high-resolution remote sensing image under complex background | |
Alidoost et al. | A CNN-based approach for automatic building detection and recognition of roof types using a single aerial image | |
CN104966085B (en) | A kind of remote sensing images region of interest area detecting method based on the fusion of more notable features | |
US9530218B2 (en) | Method for classification and segmentation and forming 3D models from images | |
CN113033520B (en) | Tree nematode disease wood identification method and system based on deep learning | |
CN111783523B (en) | Remote sensing image rotating target detection method | |
CN107067405B (en) | Remote sensing image segmentation method based on scale optimization | |
CN107103317A (en) | Fuzzy license plate image recognition algorithm based on image co-registration and blind deconvolution | |
CN108629286B (en) | Remote sensing airport target detection method based on subjective perception significance model | |
CN106846322B (en) | The SAR image segmentation method learnt based on curve wave filter and convolutional coding structure | |
CN108427919B (en) | Unsupervised oil tank target detection method based on shape-guided saliency model | |
CN103927511A (en) | Image identification method based on difference feature description | |
CN109409240A (en) | A kind of SegNet remote sensing images semantic segmentation method of combination random walk | |
CN106611423B (en) | SAR image segmentation method based on ridge ripple filter and deconvolution structural model | |
CN107545571A (en) | A kind of image detecting method and device | |
CN108090485A (en) | Display foreground extraction method based on various visual angles fusion | |
CN105931241A (en) | Automatic marking method for natural scene image |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |