CN105959521A - Method and system for improving image frame frequency based on double-camera system light-splitting imaging - Google Patents

Abstract

The invention discloses a method and system for improving the image frame frequency based on double-camera system light-splitting imaging, relates to the technical field of image processing, and solves problems that a conventional method for improving the image frame frequency is low in calculation precision and the video quality is affected because the conventional method for improving the image frame frequency is affected by parallax. The system comprises a main mirror, an auxiliary mirror, a first reflector, a second reflector, a beam shrinking system, two imaging lens, a light-splitting lens, two EMCCD cameras, a timing system controller, and a computer. Incident light is reflected by the main mirror and the auxiliary mirror, and then enters the beam shrinking system, thereby enabling the exit pupil of the beam shrinking system to be located on the front surface of the light-splitting lens. Moreover, the diameter of a light beam is matched with the size of the light-splitting lens. The incident light is divided by the light-splitting lens into reflected light and transmitting light, and the reflected light and transmitting light respectively pass through a first imaging lens group and a second imaging lens group to be received by the first EMCCD camera and the second EMCCD camera. The system employs a time sequence controller to externally trigger the two cameras for exposure and imaging in an alternate manner, and improves the sampling frame frequency of a reconstructed video image.

Description

Method and system for improving image frame rate based on dual-camera system spectroscopic imaging

technical field

The invention relates to the technical field of image processing, in particular to a method and system for improving image frame rate by using dual-camera spectroscopic imaging.

Background technique

At present, with the rapid development of digital image equipment, EMCCD cameras are playing an increasingly important role in low-light detection, adaptive optics, photoelectric countermeasures and other fields. EMCCD cameras are often difficult to balance high frame rate and high resolution. In order to improve The frame rate of the image and other performances of the EMCCD camera will inevitably be affected, and the cost of the equipment will also be greatly increased.

Chinese patent CN 201410274363.4 discloses a method for improving image frame rate and resolution based on multi-cameras, which uses a multi-camera structure-like compound eye imaging system and low-resolution video images of the same scene to reconstruct high-resolution images and improve the frame rate of video images method, but this method requires more than seven independent cameras to form a tightly arranged array. Due to the influence of parallax, a large number of image registration calculations are required, and the calculation accuracy cannot be guaranteed. Geometric errors will cause video jitter and seriously affect video quality.

Contents of the invention

The present invention provides a method and system for improving image frame rate based on dual-camera system spectroscopic imaging to solve the problem that existing methods for increasing image frame rate are affected by parallax, resulting in low calculation accuracy and further affecting video quality.

A method for improving image frame rate based on dual-camera system spectroscopic imaging, the method is implemented by the following steps:

Step 1. The incident light is reflected by the primary mirror, the secondary mirror, the first mirror and the second mirror and enters the beam reduction system. The exit pupil of the beam reduction system is located on the front surface of the beam splitter, and the beam diameter is the same as the size of the beam splitter Matching, the incident light after the beam splitter is divided into reflected light and transmitted light, the reflected light is received by the first EMCCD camera after passing through the first imaging mirror group and the second imaging mirror group, and the transmitted light is received by the second imaging mirror group Received by the second EMCCD camera;

Step 2, timing controller outputs two-way control signal, and the triggering interval of described two-way signal is half of the first EMCCD camera and the second EMCCD camera sampling period, and described timing controller controls the first EMCCD camera and the second EMCCD successively Camera exposure obtains two image sequences; The sampling period of the first EMCCD camera and the second EMCCD camera is identical;

Step 3, the computer receives the two image sequences described in step 2, uses the spatial transformation relationship to carry out feature point matching, and uses one of the image sequences of the two image sequences obtained in step 2 as a benchmark to calculate the transformation matrix, and the other image sequence performing registration, and stretching the gray scale range of the other image sequence to be the same as the gray scale range of one of the image sequences, reorganizing the two image sequences according to the time sequence relationship, and obtaining a high frame rate image sequence for performing output.

A system based on dual-camera system spectroscopic imaging to improve image frame rate, including primary mirror, secondary mirror, first mirror, second mirror, beam reduction system, two imaging lenses, beam splitter, two EMCCD cameras, time system control The incident light is reflected by the primary mirror, the secondary mirror, the first reflector and the second reflector and enters the beam reduction system. The exit pupil of the beam reduction system is located on the front surface of the beam splitter, and the beam diameter is the same as The size is matched, and the incident light is divided into reflected light and transmitted light after passing through the beam splitter. The reflected light is imaged by the first imaging mirror group and received by the first EMCCD camera, and the transmitted light is imaged by the second imaging mirror group and then received by the second EMCCD camera. The camera receives: the timing controller respectively controls the first EMCCD camera and the second EMCCD camera to start, and transmits the images acquired by the first EMCCD camera and the second EMCCD camera to the computer.

The beneficial effect of the present invention is that the two EMCCD cameras have no parallax by adopting the method of energy splitting, and there is no need to carry out a large number of image registration calculations under the condition of ensuring the accuracy of assembly and adjustment. The geometric error of the image is corrected by secondary image registration. The gray scale inconsistency of the two sequences of images caused by the different response of the EMCCD camera can be compensated by gray scale stretching. The timing controller is used to interleave and externally trigger the exposure imaging of the two cameras, which improves the reconstruction of video images. sampling frame rate.

Description of drawings

Fig. 1 is the structural schematic diagram of the system in the method for improving image frame rate based on dual-camera system spectroscopic imaging according to the present invention;

FIG. 2 is a schematic diagram of a calibration plate in the method for increasing image frame rate based on dual-camera system spectroscopic imaging according to the present invention.

detailed description

Specific Embodiments 1. This embodiment is described in conjunction with FIG. 1 and FIG. 2, a method for improving image frame rate based on dual-camera system spectroscopic imaging, which is implemented by the following steps:

1. The incident light enters the beam reduction system 5 after being reflected by the primary mirror 1 and the secondary mirror 2, the first mirror 3 and the second mirror 4, and the exit pupil of the beam reduction system 5 is located on the front surface of the beam splitter 6, and The beam diameter matches the size of the beam splitter 6, and the incident light after the beam splitter 6 is divided into reflected light and transmitted light, and the reflected light is passed through the first imaging mirror group 7 and the second imaging mirror group 8 by the first EMCCD camera 9 Receiving, the transmitted light is received by the second EMCCD camera 10 after passing through the second imaging mirror group 8;

2. Control dual cameras to shoot images of the same scene sequence: set the sampling period of the two cameras to be consistent, use two external signals to trigger, and the trigger interval for the two external signals is half the sampling period of the camera, so that the timing controller 11 Sequentially control and start two camera exposures to obtain two image sequences;

Three, the reconstruction method of the high frame rate image sequence: the first EMCCD camera 9 and the second EMCCD camera 10 take the calibration plate respectively, and carry out image binarization, and extract the outline of the white part respectively to all bright spots in the binarized image , fit a circular area, and use the image position of the center of the circular area as a matching feature point; perform feature point matching according to the spatial transformation relationship, take image sequence 1 as a benchmark, calculate the transformation matrix, and match image sequence 2 Accurate, and stretch its gray range to be the same as image sequence 1, and finally reorganize the two sequence images according to the time sequence relationship for output.

The beam splitter 6 described in this embodiment adopts a prism to split light or a flat plate to split light, and the energy ratio of reflected light and transmitted light is 1:1.

In Step 3 of this embodiment, the two cameras photograph the calibration board respectively. The background of the calibration board used is black and contains 6x6 white dot patterns, and the center of the circle is used as the matching feature point for image registration.

In Step 3 described in this embodiment, the process of reorganizing the two sequences of images according to the time series relationship is as follows: firstly determine the shooting time of the two image sequences, the image sequence with the earlier shooting time corresponds to the odd frame of the recombined image sequence, and the shooting time is at The latter image sequence corresponds to the even-numbered frames of the recombined image sequence.

Specific implementation mode 2. This implementation mode is an embodiment of the method for improving image frame rate based on dual-camera system spectroscopic imaging described in specific implementation mode 1:

1. Configure the whole system, which includes primary mirror 1, secondary mirror 2, first mirror 3, second mirror 4, beam reduction system 5, two imaging lenses, beam splitter 6, two EMCCD cameras, time system Controller 11 and computer 12, described beamsplitter 6 divides the light that is incident on spectroscopic element into reflected light and transmitted light; Described first EMCCD camera 9 is arranged on the optical path where reflected light is located, and second EMCCD camera 10 is arranged on The optical path where the transmitted light is located; the first EMCCD camera 9 and the second EMCCD camera 10 are connected with the timing controller 11 and the computer 12; the position of the beam splitter and the two cameras is adjusted so that the coverage of the field of view of the two cameras is the same. The incident light is reflected by the primary mirror 1, the secondary mirror 2, the first mirror 3 and the second mirror 4 and then enters the beam reduction system 5, so that the exit pupil of the beam reduction system 5 is located on the front surface of the beam splitter 6, and the beam diameter is the same as The size of the beam splitter is matched, and the incident light is divided into reflected light and transmitted light through the beam splitter 6, and is received by the first EMCCD camera 9 and the second EMCCD camera 10 after passing through the first imaging mirror group 7 and the second imaging mirror group 8 respectively; The system structure is shown in Figure 1.

2. Set the sampling period of the two cameras to be consistent, the imaging system is triggered by an external signal, and the triggering interval of the two signals is half the sampling period of the camera, so that the timing controller 11 sequentially controls the exposure of the two cameras to obtain the image sequence 1 and ImageSequence2;

3. Observe the consistency of the grayscale and field of view of the two image sequences. If the consistency is poor, it means that the adjustment accuracy of the optical-mechanical system does not meet the requirements, which will cause flickering and jittering of the synthesized high frame rate image sequence 3 , affecting the composite effect, at this time it is necessary to carry out geometric registration and gray scale stretching of the two sequences of images. Methods as below:

The two cameras photographed the black-and-white grid calibration board respectively. The background of the calibration board is black, and 6x6 white circles are uniformly distributed on the board, as shown in Figure 2. For each image, the image binarization is first performed, and the For all bright spots, extract the outline of the white part respectively, fit the circular area, and find the centroid position as the matching feature point, take the first EMCCD camera image as the reference image, set the second EMCCD camera image as the image to be registered, according to the space transformation The relationship between feature points is matched, the transformation matrix of the two image sequences is calculated, and the two image sequences after registration are obtained. The spatial transformation is specifically:

Assuming that the first EMCCD camera 9 obtained image is a, the pixel point coordinates are (x, y), and the second EMCCD camera 10 obtained image is b, and the pixel point coordinates are (x ', y'), then this transformation can be expressed as:

x'=r(x,y)

y'=s(x,y)

In the formula, r(x, y) and s(x, y) are space transformations, assuming that the space transformation is represented by a linear model, then:

x'=r(x,y)=c ₁ x+c ₂ y+c ₃ xy+c ₄

y'=s(x,y)=c ₅ x+c ₆ y+c ₇ xy+c ₈

There are eight unknowns in the above equations, which can be obtained only by selecting more than eight control points to solve the equations. The selection strategy of control points should cover the entire frame as much as possible to improve the solution accuracy of the model coefficients.

In this embodiment, the two image sequences are rearranged according to the time axis to generate a new image sequence. For example, if image sequence 1 is exposed first, then image sequence 1 corresponds to the odd frames of image sequence 3, and image sequence 2 corresponds to the frames of image sequence 3. If there are even frames, the frame rate of image sequence 3 thus obtained is twice that of image sequence 1 or image sequence 2, which achieves the purpose of increasing the image frame rate.

Claims (9)

1. based on dual camera systems spectroscopic imaging improve picture frame frequency method, it is characterized in that, the method by with Lower step realizes:

Step one, incident illumination are through primary mirror (1), secondary mirror (2), the first reflecting mirror (3) and the second reflecting mirror (4) entering contracting beam system (5) after reflection, the emergent pupil of described contracting beam system (5) is positioned at spectroscope (6) Front surface, and beam diameter mates with spectroscope (6) size, the incident illumination after spectroscope (6) is divided into Reflection light and transmission light, described reflection light after the first imaging lens group (7), the second imaging lens group (8) by the One EMCCD camera (9) receive, transmission light after the second imaging lens group (8) by the 2nd EMCCD phase Machine (10) receives；

Step 2, time schedule controller (11) output two path control signal, the trigger interval of described two paths of signals It is an EMCCD camera (9) and the half in the 2nd EMCCD camera (10) sampling period, time described Sequence controller (11) controls an EMCCD camera (9) successively and the 2nd EMCCD camera (10) exposes Light obtains two width image sequences；A described EMCCD camera (9) and the 2nd EMCCD camera (10) Sampling period identical；

Two image sequences described in step 3, computer (12) receiving step two, use spatial transform relation to enter Row Feature Points Matching, on the basis of the two width image sequences wherein piece image sequence that step 2 obtains, calculates and becomes Change matrix, another width image sequence is registrated, and the tonal range of another width image sequence described is stretched To the most identical with the tonal range of described wherein piece image sequence, recombinate two width image sequences according to sequential relationship, Obtain high frame frequency image sequence to export.

2. require the described method improving picture frame frequency based on dual camera systems spectroscopic imaging according to right 1, It is characterized in that, in step one, spectroscope (6) uses prismatic decomposition or flat board light splitting, reflection light and transmission Light energy is 1:1.

3. according to the side improving picture frame frequency based on dual camera systems spectroscopic imaging described in right 1 or 2 requirement Method, it is characterised in that in step 2, the process obtaining 2 width image sequence characteristic points is: described first EMCCD camera (9) and the 2nd EMCCD camera (10) shoot scaling board respectively, and carry out image two Value, bright spots all in binary image are extracted white portion profile, matching by described computer (12) respectively Border circular areas, and using the picture position, the center of circle of described border circular areas as the characteristic point mated.

4. require the described method improving picture frame frequency based on dual camera systems spectroscopic imaging according to right 3, It is characterized in that: in described step 3, the scaling board background of employing is black, comprises 6x6 white round dot Pattern, carries out image registration using home position as the characteristic point of coupling.

5. require the described method improving picture frame frequency based on dual camera systems spectroscopic imaging according to right 4, It is characterized in that: in described step 3, according to the recombinate process of two sequence images of sequential relationship be: the most true The shooting time of fixed two width image sequences, the preceding image sequence of shooting time is corresponding to reconstructed picture sequence Odd-numbered frame, shooting time corresponds to the even frame of reconstructed picture sequence at another width image sequence rear.

6. according to described in claim 1 claim based on dual camera systems spectroscopic imaging improve picture frame frequency The system of method, it is characterized in that, including primary mirror (1), secondary mirror (2), the first reflecting mirror (3), second Reflecting mirror (4), contracting beam system (5), two imaging lens, spectroscope (6), two EMCCD cameras, Time system controller (11) and computer (12), through primary mirror (1), secondary mirror, (2, first reflects incident illumination Contracting beam system (5), going out of described contracting beam system (5) is entered after mirror (3) and the second reflecting mirror (4) reflection Pupil is positioned at the front surface of spectroscope (6), and beam diameter mates with spectroscope size, through spectroscope (6) Rear incident illumination is divided into reflection light and transmission light, described reflection light after the first imaging lens group (7) imaging by first EMCCD camera (9) receive, transmission light after the second imaging lens group (8) imaging by the 2nd EMCCD phase Machine (10) receives；Described time schedule controller (11) controls an EMCCD camera (9) and second respectively EMCCD camera (10) starts, and by an EMCCD camera (9) and the 2nd EMCCD camera (10) The image obtained is sent to computer (12).

System the most according to claim 6, it is characterised in that described spectroscope (6) uses prism Light splitting or flat board light splitting, reflection light is 1:1 with transmission light energy.

System the most according to claim 6, it is characterised in that described time schedule controller (11) exports The trigger interval of two paths of signals is an EMCCD camera (9) and the sampling of the 2nd EMCCD camera (10) The half in cycle.

System the most according to claim 6, it is characterised in that a described EMCCD camera (9) Identical with the sampling period of the 2nd EMCCD camera (10).