CN105959514B - A kind of weak signal target imaging detection device - Google Patents

Abstract

The invention discloses a weak target imaging detection device and method. Utilizing the difference in light intensity between the target and the background reflected and scattered light in a specific wavelength band and in the 0° and 90° polarization directions, a dual-channel orthogonal differential imaging method is used to achieve spectral-polarization simultaneous imaging. The hardware module can be divided into three parts: the instrument shell, the optical system and the FPGA main control board. The instrument shell is used to connect the optical lens, circuit board and tripod; the optical system adopts a dual-channel structure to obtain two images with different polarization angles and wave bands; the FPGA main control board is used to configure the parameters of the dual-channel CMOS image sensor , synchronous acquisition, image caching and preprocessing. The software module performs the tasks of dual-channel image acquisition, image distortion correction, dual-channel image registration, image differential fusion and image target detection in sequence. Compared with existing methods, it has lower hardware cost and software complexity, and provides an effective means for detecting moving stealth targets in complex ground backgrounds.

Description

A Weak Target Imaging Detection Device

technical field

The invention relates to an optical imaging detection device and method, in particular to a weak target imaging detection device and method, belonging to the field of optical imaging.

Background technique

Target detection and recognition technology refers to the non-contact measurement of fixed or moving targets, and can accurately obtain the attribute information of the target and identify the authenticity of the target. Among them, optical detection has been developed rapidly and paid great attention to in recent years because it is passive and safe and hidden. However, the use of traditional camouflage paint enables the approximate realization of "same color and same spectrum" between the target and the background, and it is difficult to effectively detect "stealth" weak targets in complex backgrounds by using traditional light intensity detection methods.

Polarization is one of the basic characteristics of light. Any object will show polarization characteristics determined by its own characteristics and the basic laws of optics in the process of reflecting and emitting electromagnetic radiation. Generally, the polarization degree of the object background in the natural environment is low, while the polarization degree of the artificial target is high. For example, the degree of polarization of plants is generally less than 0.5%; the degree of polarization of rocks, sandstones, bare soil, etc. is between 0.5% and 1.5%; the degree of polarization of water surfaces, cement pavements, roofs, etc. The degree of polarization reaches 8% to 10%); the degree of polarization on the surface of some non-metallic materials and some metal materials reaches more than 2% (some even reach more than 10%). Obtaining the information of the scene in different polarization states through imaging can effectively distinguish the target and background with polarization-light intensity difference, and then realize the detection and recognition of weak targets in complex backgrounds. Therefore, in recent years, polarization imaging detection has received more and more attention in meteorological environmental scientific research, ocean development and utilization, space detection, biomedicine, and military applications.

In polarization detection, the polarization state of the target light radiation can be fully described by four Stokes parameters, including the total intensity I of the light wave, the polarization intensity Q of the line in the horizontal direction, and the polarization of the line in the 45°/135° direction The intensity U of light, and the intensity V of circularly polarized light. In practical applications, V can be ignored, and then the degree of polarization can be described as The polarization angle is described as θ=0.5 arctan(U/Q). Therefore, in order to obtain the above polarization state information, at least three light intensity images with different polarization directions must be obtained to calculate the parameters I, Q, and U.

According to this principle, there are mainly four types of polarization imaging detection devices currently in use: (1) Time-sharing imaging. This method uses an imaging device to obtain images with three different polarization directions of 0°, 60°, and 90° by sequentially rotating the polarizer installed in front of the lens; it has the advantages of simple structure and easy implementation; but it is only applicable to the target and The background is all static. (2) The way of light path splitting. This method uses a beam splitter and a retarder to divide the beam passing through the single lens into the same three parts evenly, and project it onto three independent imaging devices through polarizers in the directions of 0°, 60°, and 90°; it can simultaneously Polarization images in three directions are obtained; however, this method will greatly reduce the energy incident on a single imaging device, resulting in a significant decrease in the imaging signal-to-noise ratio. (3) The way of dividing the focal plane. This method uses an imaging device made by a special process, and each pixel on it corresponds to a polarization direction of 0°, 60°, and 90°, and is arranged in a Bayer format similar to the RGB distribution in a color image sensor; not only can Simultaneous polarization imaging can be achieved without additional optical splitting devices, and the miniaturization of the instrument can be easily realized; however, the manufacturing process of the split focal plane device is complicated and has not been commercialized. (4) The way of spatial registration. This method uses three cameras to form a three-channel synchronous imaging system, respectively collects polarization images in the directions of 0°, 60°, and 90°, and then aligns the pixels in the overlapping areas of the three images through an image space registration algorithm; it has relatively low hardware Complexity; however, due to the inconsistency of the distortion parameters and shooting angles of the three channels, if it cannot be corrected reasonably, it will lead to low image registration accuracy and affect the detection of weak and small targets. In fact, for the application of weak target detection, the purpose of polarization imaging is not to obtain polarization degree or polarization angle information, but how to enhance the contrast between the target and the background efficiently in real time. From this point of view, using the Stokes equation to fuse multi-channel images is not an efficient method.

The present invention utilizes the reflected and scattered light of the target and the background in a specific wavelength band and the difference in light intensity between 0° and 90° polarization directions, and adopts a dual-channel orthogonal differential imaging method to realize spectrum-polarization synchronous imaging. Compared with the existing synchronous The polarization imaging method has low hardware cost and software complexity, and provides an effective means for detecting moving stealth targets in complex ground backgrounds.

Contents of the invention

The invention provides a weak target imaging detection device and method aiming at the deficiencies existing in the existing moving stealth target detection system under complex ground backgrounds.

The present invention is realized through the following technical solutions:

A weak target imaging detection device is composed of three parts: an instrument shell, an optical system and an FPGA main control board. It is characterized in that: the instrument shell is used to connect the optical lens, circuit board and tripod, including the shell Rear frame and tripod mount; the optical system adopts a dual-channel structure for acquiring two images with different polarization angles and wavelength bands. Channel 1 includes a 0° linear polarizing filter, an optical lens, a C-mount lens mount, and a filter holder , 470nm narrow-band filter and CMOS image sensor; channel 2 includes 90° linear polarizing filter, optical lens, C-mount lens mount, filter holder, 630nm narrow-band filter and CMOS image sensor; for FPGA main control board It is used for parameter configuration, synchronous acquisition, image buffering and preprocessing of the dual-channel CMOS image sensor, and transmits it to the PC through the USB interface.

The size of the front panel of the housing is 100mm×50mm×5mm, on which two C-mount lens adapters for fixing the optical lens are installed, the distance between the centers of the two adapters is 50mm, and the thread outer diameter is 25.1mm; The size of the rear frame of the housing is 100mm×50mm×30mm, and it is connected to it through 12 screws with a specification of Φ3*6 around the front panel, and there is a B-type USB interface on the left side, which is used to connect the FPGA main control board and the PC machine; the tripod mount is located on the lower side of the rear frame of the casing, and is connected to the head of the tripod through a screw hole with a central specification of 1/4-20.

The focal length of the optical lens of channel 1 and channel 2 is 8mm fixed focus, the aperture adjustment range is F1.4-F16, the focus range is 0.1m-∞, and it is connected with two C-mount lens mounts on the front panel ; Two rotating linear polarizing filters are respectively installed in front of the two optical lenses through adapter rings with a size of M30.5×0.5mm; the polarization directions of the two corresponding linear polarizing filters are respectively adjusted to 0° and 90°; two narrow-band filters are respectively installed on the surface of the CMOS image sensor through the filter holder; the filters are made of mirror glass, the size is 12mm×12mm×0.7mm, and the center wavelengths are 470nm and 630nm, half bandwidth of 20nm, peak transmittance >90%, cut-off depth <1%; CMOS image sensor adopts 1.3 million pixel 1/2" monochrome area sensor, spectral response range is 400-1050nm.

The FPGA main control board takes a piece of non-volatile FPGA chip as the core, and adopts programmable system-on-chip technology to integrate a 32-bit soft-core Nios II processor and some of its peripherals into a single chip. A USB2.0 interface chip and B-type USB interface communicate with PC; Nios II processor controls user RAM, user FLASH, USB controller, two sets of dual-port RAM controllers corresponding to dual channels, and image acquisition module through Avalon bus On-chip peripherals; user RAM is used as the running memory of the Nios II processor; user FLASH is used to store the program code executed by the Nios II processor; the USB controller is used for the configuration of the USB2.0 interface chip and bus protocol conversion; the dual-port RAM is An asynchronous FIFO is used for screening and processing the effective data of the image line, and keeps the data synchronous during transmission; the image acquisition module includes two parts, the configuration controller and the timing controller, and the configuration controller passes the I ² C bidirectional data serial The buses SCLK and SDATA configure the internal registers of the CMOS image sensor, and the timing controller controls the CMOS image sensor to synchronously output data DOUT[9:0] through timing signals STROBE, PIXCLK, L_VALID, F_VALID and control signals STANDBY, TRIGGER, CLKIN.

The work flow of described FPGA main control board is: first carry out system initialization after main control board is powered on, then make Nios II processor be in wait state; After PC sends initial signal to main control board by USB interface, Nios II processes By configuring the controller, the controller sequentially writes registers to the dual-channel CMOS image sensor, sets it to capture mode, and configures parameters such as image resolution, exposure time, and electronic gain. After the setting is completed, configure the I ² C bus of the controller to enter the idle state, and make the two groups of timing controllers send TRIGGER pulses synchronously; after the CMOS image sensor receives the TRIGGER pulses, it internally resets the rows, and outputs STROBE pulses after completion, the pulse width Identify the length of pixel integration time; after the STROBE signal jumps from 1 to 0, the normal output data DOUT[7:0], and simultaneously output the synchronization signal F_VALID and L_VALID; after the timing controller receives the returned data and synchronization signal, it will first F_VALID and L_VALID perform "AND"operation; when the result is high, it means that the data is valid at this time, and then use the pixel clock as the working clock to store it in the dual-port RAM according to the address 0~1280; when the result changes from high to low, it means After a line of effective data transmission is completed, the data in the two groups of dual-port RAMs will be packaged into a data packet every 512 bytes and output to the FIFO of the USB2.0 interface chip in turn, and then transmitted to the PC via the USB cable; when a frame After the data transmission is completed, the Nios II processor sets the CMOS image sensor as the STANDBY mode through the configuration controller, stops the data output and waits for the next start signal.

As mentioned above, a detection method for a weak target imaging detection device includes the following five main steps:

(1) Dual-channel image acquisition. After the task starts, first scan the USB port and connect the specified imaging device; after confirming the connection, send the control word to the imaging device to set the imaging parameters, including image resolution, exposure time and electronic gain; complete the setting Then send a capture command and wait for image data to be received, and save the image in a lossless compressed bitmap format after the transmission of the two-channel image data is completed.

(2) Image distortion correction. The Zhang Zhengyou method is used to calibrate the optical distortion parameters of the imaging system. The nonlinear distortion model only considers the radial distortion of the image:

Among them, δ _X and δ _Y are distortion values, which are related to the pixel position of the projected point in the image. x and y are the normalized projection values obtained by the image point in the imaging plane coordinate system according to the linear projection model, k ₁ , k ₂ , k ₃ etc. are the radial distortion coefficients, only secondary distortion is considered here, and the coordinates after distortion are:

Let (u _d , v _d ) and (u, v) be the actual coordinates and ideal coordinates corresponding to the spatial points in the image coordinate system respectively, then the relationship between the two is:

The linear calibration result is used as the initial value of the parameter, and the following objective function is used to find the minimum value to realize the estimation of the nonlinear parameter:

in, is the jth point of the calibration template on the i-th image, the projection point obtained by using the estimated parameters, M _j is the coordinate value of the jth point of the calibration template in the world coordinate system, m is the number of feature points in each image, n is the number of images; use the LM iterative method to optimize the obtained camera calibration parameters, and finally obtain a more accurate radial distortion coefficient, and then inversely calculate the undistorted image coordinates.

(3) Dual-channel image registration, which is used to achieve pixel alignment of dual-channel images under different imaging fields of view, wavelength bands, polarization angles and optical distortions. An image registration algorithm based on SURF feature points is adopted, including the following five sub- step:

1) To detect SURF feature points, on the basis of constructing the integral image, use a box filter to approximately replace the second-order Gaussian filter, and calculate the Hessian value of the selected feature point and its surrounding points, if the feature point has the largest Hessian value, it is a feature point;

2) Generate a feature description vector, use the gray level information of the feature point neighborhood, and obtain the gray level distribution information to generate a 128-dimensional feature description vector by calculating the first-order Haar wavelet response of the integral image;

3) Two-step method to match feature points, through the two steps of the rough matching algorithm based on the nearest neighbor ratio method and the fine matching algorithm based on RANSAC, the correct one-to-one correspondence matching between the reference image and the feature points of the image to be registered is established relationship, which is characterized in that: after the feature vectors of the two images are generated, the Euclidean distance of the SURF feature description vector is first used as the similarity determination measure of the key points in the two images, and the method is to obtain a feature point to the nearest neighbor through the Kd tree The distance d _ND of the feature point, the distance d _NND to the next nearest neighbor feature point, if their ratio is less than the threshold ε, then keep the matching point pair formed by the feature point and its nearest neighbor; then randomly select 4 pairs of initial matching feature points, Calculate the perspective transformation matrix H determined by these 4 pairs of points, and then use this matrix to measure the matching degree of the remaining feature points:

Among them, t is the threshold, the feature point pair less than or equal to t is the inner point of H, and the feature point pair greater than t is the outer point, so that the inner point set is continuously updated, and the largest inner point can be obtained by k random sampling of RANSAC set, and the perspective transformation matrix H corresponding to the optimized interior point set is also obtained at this time;

4) Coordinate transformation and resampling. According to the obtained perspective transformation matrix H, the coordinates of the image pixels are linearly transformed, and the gray value of the image pixels is resampled by the bilinear interpolation method. The bilinear interpolation method assumes that the internal The grayscale change in the area surrounded by four points around the interpolation point is linear, so the grayscale value of the interpolated point can be calculated according to the grayscale values of the four adjacent pixels by using the linear interpolation method;

5) Crop the overlapping area of the image, judge the four boundary points after image coordinate transformation according to the following formula, and determine the four boundary point coordinates (X _min , Y _min ), (X _min , Y _{max )} of the overlapping area after image registration ), (X _max , Y _min ), (X _max , Y _max ):

Wherein, W and H are the width and height of the image, and the two-channel image is cropped according to the rectangular area formed by the above boundary points to obtain the registered 0 ° and 90 ° polarized images I (0 °) and I (90 °);

(4) Image difference fusion, the orthogonal difference image obtained by using the dual-channel orthogonal difference method is expressed as:

Q=I(0°)-I(90°)

(5) Image target detection, the system performs target detection on the orthogonal differential polarization image based on the morphological method, including the following three sub-steps:

1) Binarization processing, using the maximum inter-class variance method to adaptively select the global threshold, the principle is as follows: suppose the image has M gray values, and the value range is 0M-1, select the gray value t within this range, and set The image is divided into two groups G ₀ and G _1. The gray value of the pixels contained in G ₀ is at 0t, and the gray value of G ₁ is at t+1M-1. N represents the total number of image pixels, and n _i represents the gray value of i The number of pixels, then the probability of occurrence of each gray value i is p _i =n _i /N, the probability of occurrence of G ₀ and G ₁ is The mean is Then the between-class variance is:

σ(t) ² ＝ω ₀ ω ₁ (μ ₀ -μ ₁ ) ²

The optimal threshold T is the value of t that maximizes the variance between classes, that is:

T＝argmaxσ(t) ² , t∈[0,M-1]

2) Opening operation, the opening operation is used to filter out small interference objects and obtain a more accurate target outline, which is defined as the process of first corroding and then expanding: the role of corrosion is to eliminate irrelevant details in the object, especially the edges point, so that the boundary of the object shrinks inward, and its expression is as follows:

Among them, E represents the binary image after corrosion; B represents the structural element, that is, the template, which is a graph of any shape composed of 0 or 1, and there is a center point in B, which is the center for corrosion; X It is the pixel set of the image after binary processing of the original image; the operation process is to slide the structural element B in the X image domain, and when its center point coincides with a certain point (x, y) on the X image, traverse the structure element B If each pixel is exactly the same as the corresponding pixel in the same position centered on (x, y), then the pixel (x, y) will be retained in E. For pixels that do not meet the conditions Points are eliminated, so as to achieve the effect of shrinking the boundary; dilation and erosion have the opposite effect, it expands the boundary points of the outline of the binarized object, and can fill the remaining holes in the object after segmentation, making the object complete, and its expression The formula is as follows:

Among them, S represents the set of expanded binary image pixels; B represents the structural element, that is, the template; X represents the set of image pixels after binary processing. The operation process is to slide the structural element B in the X image domain. When the center point of B moves to a certain point (x, y) on the X image, traverse the pixels in the structural element. If the pixel in the structural element B is the same as X At least one pixel of the image is the same, then keep the (x, y) pixel in S, otherwise remove this pixel; after the binary image is opened, the image is divided into multiple connected regions;

3) Connected domain identification, first use the 8-adjacent criterion to segment the connected domain in the image, the definition of the 8-adjacent connected domain is: each pixel in the area has at least 8 adjacent pixels in all 8 directions A pixel still belongs to this area, and according to the definition, different connected domains in the binary image are filled with different digital marks; then the pixel perimeters of each connected domain are extracted respectively, and compared with the preset target threshold, if in In the threshold interval, it is judged as a candidate target; finally, the candidate target is identified in the image by the smallest rectangular frame that can surround the outline of its connected domain, and the target detection is completed.

The present invention has the following beneficial effects:

1. The hardware system is easy to realize. There is no need for complex optical path splitting design or imaging device manufacturing process.

2. The computational complexity of the software is low. Complicated camera calibration only needs to be performed once in the laboratory; image fusion does not need to calculate the degree of polarization, and only needs to perform a simple pixel grayscale difference operation.

3. The registration accuracy of the algorithm is high. Camera nonlinear distortions were corrected before registration.

4. It is suitable for the detection of moving targets.

Description of drawings

Fig. 1 is a block diagram of software and hardware functional modules of the weak target imaging detection system involved in the present invention.

Fig. 2 is the three-dimensional schematic view of the hardware structure of the weak target imaging detection device involved in the present invention, in which the label names: 1 is the front panel of the housing; 2 is the rear frame of the housing; 3 is the tripod fixing seat; 4 is the 0° linear polarization filter ; 5 is 90° linear polarizing filter; 6, 7 is optical lens; 8, 9 is C-mount lens adapter ring; 10, 11 is filter seat; 12 is 470nm narrow-band filter; 13 is 630nm narrow-band filter 14, 15 are CMOS image sensors; 16 is a USB interface.

FIG. 3 is a block diagram of the hardware circuit of the FPGA main control board involved in the present invention.

Fig. 4 is a software flow diagram of the weak target imaging detection method involved in the present invention.

Detailed ways

The technical scheme of the present invention is described in detail below in conjunction with accompanying drawing:

The block diagram of software and hardware functional modules of the weak target imaging detection system of the present invention is shown in FIG. 1 . The hardware module of the weak target imaging detection device can be divided into three parts: the instrument shell, the optical system and the FPGA main control board. Among them, the instrument housing is used to connect the optical lens, circuit board and tripod, including the front panel of the housing, the rear frame of the housing and the tripod holder; the optical system adopts a dual-channel structure, which is used to obtain two images with different polarization angles and wave bands. Channel 1 includes 0° linear polarizing filter, optical lens, C-mount lens adapter, filter holder, 470nm narrow-band filter and CMOS image sensor; channel 2 includes 90° linear polarizing filter, optical lens, C-mount lens Mounting ring, filter holder, 630nm narrow-band filter and CMOS image sensor; FPGA main control board is used for parameter configuration, synchronous acquisition, image buffer and preprocessing of the dual-channel CMOS image sensor, and transmits to PC through USB interface machine. The software module runs on the PC, and performs the tasks of dual-channel image acquisition, image distortion correction, dual-channel image registration, image differential fusion and image target detection in sequence.

A three-dimensional schematic diagram of the hardware structure of the weak target imaging detection device of the present invention is shown in FIG. 2 . The size of the front panel 1 of the housing is 100mm×50mm×5mm, on which are mounted C-mount lens adapters 8 and 9 for fixing the optical lens, the distance between the centers of the two adapters is 50mm, and the thread outer diameter is 25.1mm. The size of the rear frame 2 of the housing is 100mm×50mm×30mm, and it is connected to it through 12 screws with a specification of Φ3*6 around the front panel; there is a B-type USB interface 16 on the left side, which is used to connect to the FPGA main control board and PCs. The tripod mount 3 is positioned at the lower side of the housing rear frame, and is connected to the cloud platform of the tripod through a screw hole of 1/4-20 (outer diameter 1/4 inch, pitch 20 teeth/inch) through the central specification. The focal lengths of optical lenses 6 and 7 are both 8mm fixed focus, the aperture adjustment range is F1.4-F16, and the focus range is 0.1m-∞, which are respectively connected to the C-mount lens mounts 8 and 9 on the front panel. Two rotating linear polarizing filters 4 and 5 are respectively installed in front of the optical lens 6 and 7 through an adapter ring with a size of M30.5×0.5mm (outer diameter 30.5mm, pitch 0.5mm); The polarization directions of the corresponding linear polarizing filters were adjusted to 0° and 90°, respectively. The filter holders 10 and 11 through which the two narrow-band filters 12 and 13 respectively pass are installed on the surfaces of the CMOS image sensors 14 and 15; the filters are all made of mirror glass, and the size is 12mm×12mm×0.7mm, and the center wavelength 470nm and 630nm respectively, the half-bandwidth is 20nm, the peak transmittance is >90%, and the cut-off depth is <1%. Both the CMOS image sensors 14 and 15 are MT9M001 with 1.3 million pixels. MT9M001 is a 1/2″ monochromatic area sensor with a spectral response range of 400-1050nm; the imaging signal-to-noise ratio and dynamic range are 45dB and 68.2dB respectively, which can reach the level of CCD; the pixel size of 5.2μm×5.2μm It achieves a high low-light sensitivity of 2.1V/lux-sec; and the continuous image capture capability of 1280×1024@30fps can meet the detection requirements of most moving targets.

The hardware circuit block diagram of the FPGA main control board of the present invention is as shown in FIG. 3 . In order to realize the synchronous acquisition and control of the dual-channel CMOS image sensor, the hardware design of the main control board takes a non-volatile FPGA chip as the core, and adopts the programmable system-on-chip technology to integrate the 32-bit soft-core Nios II processor and its parts Peripherals are integrated in a single chip, and only one USB2.0 interface chip and B-type USB interface are used outside the chip to communicate with the PC, which greatly improves the integration of system component functions and reduces system-level costs. The Nios II processor is built in the form of an IP core, and controls user RAM, user FLASH, USB controller, two sets of dual-port RAM controllers corresponding to dual channels, and image acquisition modules and other on-chip peripherals through the Avalon bus. Among them, the user RAM is used as the running memory of the Nios II processor; the user FLASH is used to store the program code executed by the Nios II processor; the USB controller is used for the configuration of the USB2.0 interface chip and the conversion of the bus protocol; the dual-port RAM is an asynchronous FIFO is used for screening and processing the effective data of the image line, and keeps the data synchronous during the transmission process; the image acquisition module includes two parts, the configuration controller and the timing controller, and the configuration controller passes through the I ² C bidirectional data serial bus SCLK , SDATA configure the internal registers of the CMOS image sensor, and the timing controller controls the CMOS image sensor to synchronously output data DOUT[9:0] through timing signals STROBE, PIXCLK, L_VALID, F_VALID and control signals STANDBY, TRIGGER, CLKIN.

During specific implementation, the FPGA chip adopts the MAX 10 series chip of ALTERA Company whose model number is 10M08E144ES. This chip is manufactured using TSMC's 55nm embedded NOR flash memory technology, with 8K logic units, 378Kb embedded SRAM resources, and 172KB user FLASH resources. Since the maximum pixel array of the CMOS image sensor is 1280×1024, the number of quantization bits is 8bit, and a storage space of 10Kbit is required for caching one line of data, two pieces of 10Kb space are allocated from the embedded SRAM resource to build two dual-port RAM, while assigning the remaining 358Kb to user RAM. The USB2.0 interface chip adopts CY7C68013A of CYPRESS company, and its internal FIFO resource size is 4KB. Peripheral equipment and USB interface can operate on this FIFO resource at the same time. Data transmission between FIFO and external circuit can be carried out without the participation of USB firmware program. , the maximum transfer rate is 96MB/s.

The workflow of the FPGA main control board is as follows: After the main control board is powered on, the system is initialized first, and then the Nios II processor is in a waiting state. After the PC sends the start signal to the main control board through the USB interface, the Nios II processor writes registers to the dual-channel CMOS image sensor in turn through the configuration controller, sets it as the capture mode, and configures the image resolution and exposure time And electronic gain and other parameters. After the setting is completed, configure the I ² C bus of the controller to enter the idle state, and make the two groups of timing controllers send TRIGGER pulses synchronously. After the CMOS image sensor receives the TRIGGER pulse, it internally resets the row, and outputs a STROBE pulse after completion, and the pulse width indicates the length of the pixel integration time. After the STROBE signal changes from 1 to 0, the data DOUT[7:0] is normally output, and the synchronization signals F_VALID and L_VALID are output at the same time. After the timing controller receives the returned data and synchronous signal, it first performs "AND" operation on F_VALID and L_VALID. When the result is high, it means that the data is valid at this time, and then the pixel clock is used as the working clock to store it in the dual-port RAM according to the address 0~1280; The data in the two groups of dual-port RAM is packed into a data packet every 512 bytes and output to the FIFO of the USB2.0 interface chip in turn, and then transmitted to the PC via the USB line. After a frame of data transmission is completed, the Nios II processor sets the CMOS image sensor to STANDBY mode through the configuration controller, stops data output and waits for the next start signal.

The software flow diagram of the weak target imaging detection method of the present invention is shown in FIG. 4 . The weak target imaging detection method includes the following five main steps:

(1) Two-channel image acquisition. After the task starts, first scan the USB port and connect the specified imaging device; after confirming the connection, send a control word to the imaging device to set the imaging parameters, including image resolution, exposure time and electronic gain; after completing the setting, send an acquisition command and wait Receive the image data, and save the image in a lossless compressed bitmap format after the transmission of the two-channel image data is completed.

(2) Image distortion correction. In order to achieve accurate registration of dual-channel images, it is necessary to perform distortion correction on the two images separately. Considering that the dual channels of the imaging system are independent, the design adopts the classic Zhang Zhengyou plane calibration method to calibrate the optical distortion parameters of the imaging system. Optical distortion is nonlinear, mainly including radial distortion, tangential distortion, centrifugal distortion and thin prism distortion, etc. It is necessary to use a nonlinear model to estimate the distortion parameters. Among them, radial distortion is the main factor of image error, and its model can be approximately described as:

Among them, δ _X and δ _Y are distortion values, which are related to the pixel position of the projected point in the image. x and y are the normalized projection values obtained by the image point in the imaging plane coordinate system according to the linear projection model, k ₁ , k ₂ , k ₃ etc. are the radial distortion coefficients, only secondary distortion is considered here, and the coordinates after distortion are:

Let (u _d , v _d ) and (u, v) be the actual coordinates and ideal coordinates corresponding to the spatial points in the image coordinate system, respectively. Then the relationship between the two is:

The linear calibration result is used as the initial value of the parameter, and the following objective function is used to find the minimum value to realize the estimation of the nonlinear parameter:

in, is the jth point of the calibration template on the i-th image, the projection point obtained by using the estimated parameters, M _j is the coordinate value of the jth point of the calibration template in the world coordinate system, m is the number of feature points in each image, n is the number of images. The LM iterative method is used to optimize the obtained camera calibration parameters, and finally a more accurate radial distortion coefficient is obtained, and then the undistorted image coordinates are inversely calculated.

(3) Two-channel image registration. Due to the difference in the imaging field of view, wavelength band, polarization angle and optical distortion of the two channels, the two images need to be registered to align the pixels to be fused. Considering that SURF feature points are robust to image rotation, translation, scaling and noise, an image registration algorithm based on SURF feature points is adopted, including the following five sub-steps:

1) To detect SURF feature points, on the basis of constructing the integral image, use a box filter to approximately replace the second-order Gaussian filter, and calculate the Hessian value of the selected feature point and its surrounding points, if the feature point has the largest Hessian value, it is a feature point.

2) To generate the feature description vector, the gray level information of the feature point neighborhood is used, and the gray level distribution information is obtained by calculating the first-order Haar wavelet response of the integral image to generate a 128-dimensional feature description vector.

3) Two-step method to match feature points, through the two steps of the rough matching algorithm based on the nearest neighbor ratio method and the fine matching algorithm based on RANSAC, the correct one-to-one correspondence matching between the reference image and the feature points of the image to be registered is established relation. It is characterized in that: after the feature vectors of the two images are generated, the Euclidean distance of the SURF feature description vector is first used as the similarity judgment measure of the key points in the two images, and the method is to obtain a feature point to the nearest neighbor feature point through the Kd tree d _ND , and the distance d _NND to the next nearest neighbor feature point, if their ratio is less than the threshold ε, the matching point pair consisting of the feature point and its nearest neighbor is retained; then 4 pairs of initial matching feature points are randomly selected, and the calculation is given by The perspective transformation matrix H determined by these 4 pairs of points is then used to measure the matching degree of the remaining feature points:

Among them, t is the threshold value, the feature point pair less than or equal to t is the inner point of H, and the feature point pair greater than t is the outer point. In this way, the interior point set is continuously updated, and the largest interior point set can be obtained by k random sampling of RANSAC. At this time, the perspective transformation matrix H corresponding to the optimized interior point set is also obtained.

4) Coordinate transformation and resampling. According to the obtained perspective transformation matrix H, the coordinates of the image pixels are linearly transformed, and the gray value of the image pixels is resampled by bilinear interpolation method. The bilinear interpolation method assumes that the gray level change in the area surrounded by four points around the interpolation point is linear, so that the gray value of the interpolation point can be calculated according to the gray value of the four adjacent pixels using the linear interpolation method. degree value.

5) Cropping image overlap area. Discriminate the four boundary points after the image coordinate transformation according to the following formula, and determine the four boundary point coordinates (X _min , Y _min ), (X _min ,Y _max ), (X _max ,Y _min ), (X _max ,Y _max ):

Among them, W and H are the width and height of the image. The two-channel image was cropped according to the rectangular area formed by the above boundary points, and the registered 0° and 90° polarization images I(0°) and I(90°) were obtained.

(4) Image differential fusion. Since the reflected and scattered light of the target and the background has a significant difference in light intensity in the 0° and 90° polarization directions, the dual-channel orthogonal differential image fusion method can not only obtain a better image signal-to-noise ratio, but also has an extremely low software complexity. The fused orthogonal difference image is expressed as:

Q=I(0°)-I(90°) (7)

(5) Image target detection. Mathematical morphology is a mathematical method for analyzing geometric shapes and contour structures of objects, mainly including expansion, erosion, opening and closing operations, etc. In the field of image processing, it is used to "maintain the basic shape of the object and remove irrelevant features", and can extract features useful for expressing and describing the shape. Usually, morphological processing is performed as a template-based neighborhood operation method, that is, to define a special neighborhood called a "structural element" or template, and place It performs some logical operation on the area corresponding to the binary image, and the result is the pixel value of the output image. The size, content and operation properties of structural elements will affect the results of morphological processing. The system performs target detection on orthogonal differential polarization images based on the morphology method, which has the characteristics of clear physical meaning and high operation efficiency, including three sub-steps of image binarization, opening operation, and connected domain identification.

1) Binary processing. Image binarization is the premise of morphological filtering, and selecting an appropriate segmentation threshold is an important step. Here, the maximum inter-class variance method is used to adaptively select the global threshold. This algorithm was proposed by Otsu in 1979. It is based on the statistical characteristics of the entire image to realize the automatic selection of the threshold. It is the most outstanding representative of global binarization. The basic idea of the algorithm is to use a certain assumed gray value to divide the gray value of the image into two groups. When the variance between the two groups is the largest, this gray value is the optimal threshold for image binarization. Assuming that the image has M gray values, the value range is 0 M-1, select the gray value t within this range, divide the image into two groups G ₀ and G ₁ , the gray value of the pixels contained in G ₀ is 0 t, the gray value of G ₁ is at t+1 M-1, use N to represent the total number of image pixels, n _i represents the number of pixels with gray value i, then the probability of occurrence of each gray value i is p _i ＝n _i /N, the probability of G ₀ and G ₁ class appearing is The mean is Then the between-class variance is:

σ(t) ² ＝ω ₀ ω ₁ (μ ₀ -μ ₁ ) ² (8)

The optimal threshold T is the value of t that maximizes the variance between classes, that is:

T＝argmaxσ(t) ² ,t∈[0,M-1] (9)

2) Open operation. The opening operation is used to filter out small interference objects and obtain a more accurate target outline. It is defined as the process of first erosion and then expansion: the main function of erosion is to eliminate irrelevant details in the object, especially the edge points, so that the boundary of the object shrinks inward. Its expression is as follows:

Among them, E represents the binary image after corrosion; B represents the structural element, that is, the template, which is a graph of any shape composed of 0 or 1, and there is a center point in B, which is the center for corrosion; X It is a collection of pixels of the image after the original image has been binarized. The operation process is to slide the structural element B in the X image domain. When its center point coincides with a certain point (x, y) on the X image, traverse the pixels in the structural element. The corresponding pixels in the same position centered on y) are exactly the same, then the pixel (x, y) will be kept in E, and the pixels that do not meet the conditions will be eliminated, so as to achieve the effect of shrinking the boundary. Expansion is the opposite of erosion. It expands the boundary points of the binary object outline, which can fill the remaining holes in the segmented object and make the object complete. Its expression is as follows:

Among them, S represents the set of expanded binary image pixels; B represents the structural element, that is, the template; X represents the set of image pixels after binary processing. The operation process is to slide the structural element B in the X image domain. When the center point of B moves to a certain point (x, y) on the X image, traverse the pixels in the structural element. If the pixel in the structural element B is the same as X If at least one pixel of the image is the same, then keep the (x, y) pixel in S, otherwise remove this pixel.

3) Connected domain identification. After the binary image is opened, the image is divided into multiple connected regions. In order to screen out candidate targets, the connected domains need to be segmented, labeled, and features extracted for target recognition. The purpose of connected domain segmentation is to extract the target "1" value pixel set adjacent to each other in a lattice binary image, and fill in different digital marks for different connected domains in the image. Algorithms are usually divided into two categories: one is the local neighborhood algorithm, the basic idea is to check each connected component one by one from the local to the whole, determine a "starting point", and then fill in the markers to the surrounding neighborhood; the other The class is from the whole to the part, first determine the different connected components, and then fill in the mark with the method of area filling for each connected component. Here, the 8-neighbor criterion is used to search and mark the connected domains in the image. The definition of an 8-adjacent connected domain is that for each pixel in the region, at least one of its 8 adjacent pixels in all 8 directions still belongs to the region. After completing the segmentation and marking of connected domains, extract the pixel perimeter of each connected domain and compare them with the preset target threshold. If it is within the threshold range, it is judged as a candidate target, and the smallest rectangular frame that can surround the outline of its connected domain is adopted. Targets are identified in the image.

Claims (5)

1. The utility model provides a weak target formation of image detection device, comprises instrument casing, optical system and FPGA main control board triplex which characterized in that: the instrument shell is used for connecting the optical lens, the circuit board and the tripod and comprises a shell front panel, a shell rear frame and a tripod fixing seat; the optical system adopts a dual-channel structure and is used for acquiring two images with different polarization angles and wave bands, and a channel 1 comprises a 0-degree linear polarization filter, an optical lens, a C-port lens joint ring, a filter base, a 470nm narrow-band filter and a CMOS image sensor; the channel 2 comprises a 90-degree linear polarization filter, an optical lens, a C-port lens joint ring, a filter seat, a 630nm narrow-band filter and a CMOS image sensor; the FPGA main control board is used for carrying out parameter configuration, synchronous acquisition, image caching and preprocessing on the dual-channel CMOS image sensor and transmitting the parameters to the PC through the USB interface.

2. A weak target imaging detection device according to claim 1, characterized in that: the size of the front panel of the shell is 100mm multiplied by 50mm multiplied by 5mm, two C-port lens joint rings for fixing an optical lens are arranged on the front panel, the central distance between the two joint rings is 50mm, and the external diameter of a thread is 25.1 mm; the size of the rear frame of the shell is 100mm multiplied by 50mm multiplied by 30mm, the rear frame is connected with the front panel through 12 screws with the specification of phi 3 x 6 on the periphery of the front panel, and the left side of the rear frame is provided with a B-type USB interface used for connecting an FPGA main control board and a PC; the tripod fixing seat is positioned on the lower side of the rear frame of the shell and is connected with a tripod head of a tripod through a screw hole with the central specification of 1/4-20.

3. A weak target imaging detection device according to claim 1, characterized in that: the focal lengths of the optical lenses of the channel 1 and the channel 2 are both 8mm fixed focuses, the aperture adjusting range is F1.4-F16, the focusing range is 0.1 m-infinity, and the optical lenses are connected with the two C-port lens connecting rings on the front panel; the two rotary linear polarization filters are respectively arranged in front of the two optical lenses through connecting rings with the size of M30.5 multiplied by 0.5 mm; respectively adjusting the polarization directions of the linear polarization filters corresponding to the linear polarization calibration plate to 0 degree and 90 degrees by adopting the linear polarization calibration plate; the two narrow-band filters are respectively arranged on the surface of the CMOS image sensor through filter seats; the optical filters are made of mirror glass, the size of the optical filters is 12mm multiplied by 0.7mm, the central wavelength is 470nm and 630nm respectively, the half bandwidth is 20nm, the peak transmittance is more than 90%, and the cut-off depth is less than 1%; the CMOS image sensor adopts an 1/2' monochrome area array sensor with 130 ten thousand pixels, and the spectral response range is 400-1050 nm.

4. A weak target imaging detection device according to claim 1, characterized in that: the FPGA main control board takes a nonvolatile FPGA chip as a core and adoptsThe system technology on the programmable chip integrates a 32-bit soft core Nios II processor and partial peripheral equipment thereof into a single chip, and only one USB2.0 interface chip and a B-type USB interface are adopted outside the chip to communicate with a PC; the Nios II processor controls the on-chip peripherals such as a user RAM, a user FLASH, a USB controller, a 2-group double-port RAM controller corresponding to double channels, an image acquisition module and the like through an Avalon bus; the user RAM is used as the running memory of the Nios II processor; the user FLASH is used for storing program codes executed by the Nios II processor; the USB controller is used for the configuration and bus protocol conversion of the USB2.0 interface chip; the double-port RAM is an asynchronous FIFO and is used for screening and processing effective data of image lines and keeping the data synchronous in the transmission process; the image acquisition module comprises a configuration controller and a time schedule controller, wherein the configuration controller passes through I²C bidirectional data serial buses SCLK and SDATA configure internal registers of the CMOS image sensor, and the timing controller controls the CMOS image sensor to synchronously output data DOUT [9:0 ] through timing signals STROBE, PIXCLK, L _ VALID and F _ VALID and control signals STANDBY, TRIGGER and CLKIN]。

5. A weak target imaging detection apparatus as claimed in claim 1, wherein: the imaging detection comprises the following five main steps:

(1) the method comprises the following steps of (1) double-channel image acquisition, wherein after a task is started, a USB port is scanned and connected with a specified imaging device; after the connection is confirmed, sending a control word to the imaging device to set imaging parameters including image resolution, exposure time and electronic gain; after the setting is finished, sending a primary acquisition instruction and waiting for receiving image data, and saving the image in a lossless compression bitmap format after the transmission of the image data of the two channels is finished;

(2) correcting image distortion, designing an optical distortion parameter of the imaging system by adopting a Zhang-friend method, and considering only radial distortion of an image by a nonlinear distortion model:

wherein, delta_XAnd delta_YIs distortion value, which is related to the pixel position of the projection point in the image, x and y are normalized projection values of the image point obtained according to the linear projection model under the imaging plane coordinate system,k₁、k₂、k₃equal to the radial distortion coefficient, only the second order distortion is considered here, and the distorted coordinates are:

order (u)_d,v_d) And (u, v) are respectively an actual coordinate and an ideal coordinate corresponding to the space point under the image coordinate system, and the relationship between the actual coordinate and the ideal coordinate is as follows:

taking the linear calibration result as an initial parameter value, bringing the initial parameter value into the following objective function to solve the minimum value, and realizing the estimation of the nonlinear parameter:

wherein,is the projection point M obtained by using the estimated parameters of the jth point of the calibration template on the ith image_jThe coordinate value of the jth point of the calibration template in a world coordinate system is defined, m is the number of characteristic points of each image, and n is the number of images; optimizing the obtained camera calibration parameters by using an LM iteration method to finally obtain a more accurate radial distortion coefficient, and further reversely solving an undistorted image coordinate;

(3) the dual-channel image registration is used for realizing pixel alignment of dual-channel images under different imaging fields of view, wave bands, polarization angles and optical distortion conditions, and an image registration algorithm based on SURF feature points is adopted, and the image registration algorithm comprises the following five substeps:

1) detecting SURF characteristic points, on the basis of constructing an integral image, approximately replacing second-order Gaussian filtering by using square filtering, respectively calculating Hessian values of the characteristic points to be selected and points around the characteristic points, and if the characteristic points have the maximum Hessian value, taking the characteristic points as the characteristic points;

2) generating a feature description vector, and generating a 128-dimensional feature description vector by calculating first-order Haar wavelet response of an integral image by using gray scale information of a feature point neighborhood to obtain gray scale distribution information;

3) the two-step method is used for matching feature points, and a correct one-to-one corresponding matching relation between the reference image and the feature points of the image to be registered is established through two steps of a coarse matching algorithm based on a nearest neighbor ratio method and a precise matching algorithm based on RANSAC, and the method is characterized in that: after the feature vectors of the two images are generated, firstly, the Euclidean distance of the SURF feature description vectors is adopted as the similarity judgment measurement of key points in the two images, and the method is to obtain the distance d from one feature point to the nearest neighbor feature point through a K-d tree_NDIts distance d to next-nearest neighbor feature point_NNDIf the ratio of the characteristic points to the nearest neighbor points is smaller than the threshold epsilon, keeping the matching point pairs formed by the characteristic points and the nearest neighbor points; then, 4 pairs of initial matching feature points are randomly selected, a perspective transformation matrix H determined by the 4 pairs of the initial matching feature points is calculated, and the matching degree of the other feature points is measured by the matrix:

wherein t is a threshold value, the characteristic point pairs smaller than or equal to t are inner points of H, and the characteristic point pairs larger than t are outer points, so that an inner point set is continuously updated, the largest inner point set can be obtained by k times of random sampling of RANSAC, and a perspective transformation matrix H corresponding to the optimized inner point set is obtained;

4) performing coordinate transformation and resampling, namely performing linear transformation on the coordinates of the image pixels according to the obtained perspective transformation matrix H, and performing resampling on the gray value of the image pixels by adopting a bilinear interpolation method, wherein the bilinear interpolation method assumes that the gray change in an area surrounded by four points around the interpolation point is linear, so that the gray value of the interpolation point can be calculated by using a linear interpolation method according to the gray values of four adjacent pixels;

5) cutting the image overlapping area, judging the four boundary points after the image coordinate transformation according to the following formula, and determining the coordinates (X) of the four boundary points of the overlapping area after the image registration_min,Y_min)、(X_min,Y_max)、(X_max,Y_min)、(X_max,Y_max)：

Wherein W, H is the width and height of the image, and the two-channel image is clipped according to the rectangular region formed by the above boundary points to obtain the registered 0-degree and 90-degree polarization images I (0) and I (90);

(4) and image difference fusion, wherein an orthogonal difference image obtained by fusion in a two-channel orthogonal difference mode is represented as follows:

Q＝I(0°)-I(90°)；

(5) the image target detection is carried out on the orthogonal differential polarization image by a system based on a morphology method, and the system comprises the following three substeps:

1) and (3) binarization processing, namely adopting a maximum inter-class variance method to adaptively select a global threshold value, wherein the principle is as follows: setting M gray values of the image, wherein the value range is 0-M-1, selecting the gray value t in the range, and dividing the image into two groups G₀And G₁，G₀The gray value of the contained pixel is 0-t, G₁The gray value of (1) is t + 1-M-1, the total number of image pixels is represented by N, N_iRepresenting the number of pixels with a gray value i, the probability of each gray value i appearing is p_i＝n_i/N，G₀And G₁The probability of class occurrence isMean value ofThe between-class variance is:

σ(t)²＝ω₀ω₁(μ₀-μ₁)²

the optimal threshold T is the value of T that maximizes the inter-class variance, i.e.:

T＝argmaxσ(t)²,t∈[0,M-1]

2) and the opening operation is used for filtering fine interferent and obtaining a more accurate target profile, and is defined as a process of firstly corroding and then expanding: the effect of erosion is to eliminate irrelevant details in the object, particularly the edge points, and shrink the boundary of the object inwards, which is expressed as follows:

wherein E represents a binary image after corrosion; b represents a structural element, namely a template, which is a figure in any shape consisting of 0 or 1, and a central point is arranged in B, and the etching is carried out by taking the central point as the center; x is a pixel set of an image of an original image after binarization processing; the operation process is that the structural element B is slid in the X image domain, when the central point of the structural element B is superposed with a certain point (X, y) on the X image, pixel points in the structural element are traversed, if each pixel point is completely the same as the corresponding pixel point in the same position taking (X, y) as the center, the pixel point (X, y) is kept in E, and the pixel points which do not meet the condition are removed, so that the effect of shrinking the boundary can be achieved; the expansion and corrosion effects are opposite, the boundary point of the binary object contour is expanded, the residual holes in the divided object can be filled, the object is complete, and the expression is as follows:

s represents a set of expanded binary image pixel points; b represents a structural element, namely a template; x represents an image pixel set after binarization processing, the operation process is to slide a structural element B in an X image domain, when the central point of B is moved to a certain point (X, y) on an X image, pixel points in the structural element are traversed, if the pixel point in the structural element B is at least one same as the pixel point of the X image, the pixel point (X, y) is kept in S, otherwise, the pixel point is removed; after the binary image is subjected to open operation, the image is divided into a plurality of connected regions;

3) identifying a connected domain, namely firstly segmenting the connected domain in the image by adopting 8 adjacent criteria, wherein the definition of the 8 adjacent connected domains is as follows: filling different connected domains in the binary image with different digital marks according to the definition, wherein each pixel in the region still belongs to at least one of 8 adjacent pixels in all 8 directions; then respectively extracting the pixel circumferences of all connected domains, comparing the pixel circumferences with a preset target threshold, and judging as a candidate target if the pixel circumferences are within a threshold interval; and finally, identifying candidate targets in the image by adopting a minimum rectangular frame capable of surrounding the outline of the connected domain of the target, and completing target detection.