JP2008010958A - Imaging apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine a motion vector accurately even in case of starlit sky photography. <P>SOLUTION: The imaging apparatus comprises a motion vector detection circuit 6 for dividing both first image data outputted from an image sensor 2 having a plurality of pixels and second image data outputted from the image sensor before the first image data is outputted, respectively, into a plurality of regions and calculating a motion vector for each divided region, and an isolated region judging circuit 5 for judging an isolated region existing in each divided region wherein the motion vector detection circuit calculates a motion vector only in such a divided region where the isolated regions exceeding a predetermined number are detected by the isolated region judging circuit. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method provided with an imaging element such as a CCD image sensor.

  Recent imaging apparatuses are required to be able to photograph “starry sky” for astronomical observation. In addition, there is an increasing demand for hand-held shooting without using a tripod for easy shooting of the starry sky. For this reason, the image pickup apparatus needs to have an image stabilization function at the time of shooting the starry sky, and this image stabilization function can be realized by calculating a motion vector of an image.

  A method for calculating a motion vector is described, for example, in Patent Document 1 below. In the method described in Patent Literature 1, two temporally continuous images are divided into a plurality of vertical and horizontal directions. Subsequently, for each of the plurality of divided blocks obtained by the division, a correlation value for the amount of shift between the two divided images corresponding to each other in temporally continuous images is obtained. Then, a motion vector indicating the amount of parallel movement of the entire image is calculated based on the obtained correlation value.

JP-A 61-269475

However, when the method described in Patent Document 1 is used for starry sky photography, there are the following problems.
First, if there is no star in the divided block and the screen is black (only in the night sky), the correlation value for the amount of shift between two consecutive frames is always the maximum, and the motion vector is accurate. Can not ask for.
In addition, when there is a defective pixel called “white scratch” on the image sensor included in the image pickup device, the white image that looks like a star in only one of two temporally continuous images. A point exists and the motion vector cannot be obtained accurately.
For example, if only one star appears to blink, one of the two consecutive frames of time may not have a star. Cannot accurately determine the correlation between images.

  The present invention has been made in view of such circumstances, and an object of the present invention is to accurately obtain a motion vector even at the time of shooting a starry sky by an imaging apparatus.

An image pickup apparatus according to the present invention divides an image pickup device having a plurality of pixels and first image data output from the image pickup device into a plurality of regions, and outputs the first image data before the first image data. A motion vector detecting means for calculating a motion vector related to each divided area with reference to the second image data, and an isolated area for discriminating an isolated area existing in the divided area of the first image data for each divided area; Discriminating means, wherein the motion vector detecting means calculates a motion vector related to the divided area according to the discrimination result of the isolated area discriminating means.
The imaging method according to the present invention divides first image data output from an imaging device having a plurality of pixels into a plurality of regions, and outputs second image data prior to the first image data. A motion vector detecting step for calculating a motion vector relating to each divided region, and an isolated region determining step for determining for each divided region an isolated region existing in the divided region of the first image data. The motion vector relating to the divided region is calculated in the motion vector detecting step in accordance with the discrimination result in the isolated region discriminating step.
The program according to the present invention divides first image data output from an imaging device having a plurality of pixels into a plurality of regions, and outputs second image data output before the first image data. A motion vector detecting step for calculating a motion vector related to each divided region with reference to the computer, and an isolated region determining step for determining an isolated region existing in the divided region of the first image data for each divided region. In the motion vector detection step, the motion vector relating to the divided region is calculated according to the determination result in the isolated region determination step.

  According to the present invention, the image data from the image sensor is divided into a plurality of areas, the number of isolated areas is obtained for each divided area, and the motion vector is calculated only in the divided areas where a predetermined number or more of isolated areas exist. Thus, the motion vector can be accurately obtained even during starry sky photography.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
In the embodiment of the present invention described below, in view of the above-described conventional problems, an isolated region corresponding to a star is obtained when obtaining a correlation value related to the shift amount between two temporally continuous images. The number is obtained, and the motion vector is calculated only in the divided area where there are a predetermined number or more of isolated areas. In this way, the motion vector is accurately obtained even when shooting a starry sky, thereby realizing an accurate image stabilization operation.

(First embodiment)
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to the first embodiment of the present invention.
In FIG. 1, an input signal that has passed through a lens 1 and is photoelectrically converted by an image sensor 2 is subjected to processing such as reset noise removal, gain adjustment, and analog-digital (AD) conversion by an analog front end (AFE) circuit 3. And converted into digital image data. The imaging device 2 has a plurality of pixels, and for example, a CCD (charge coupled device) image sensor or a CMOS image sensor is used.

  The digital image data that is the output of the AFE circuit 3 is supplied to the camera signal processing circuit 4 and subjected to imaging system camera signal processing such as aperture correction, gamma correction, and white balance. The output of the camera signal processing circuit 4 is stored in the memory 7 for one frame or one field according to the drive mode of the image sensor 2. Here, the memory control circuit 8 controls the memory 7 in accordance with an instruction or the like supplied via the data bus 14.

  The output from the camera signal processing circuit 4 and the output from the memory 7 are supplied to the defective pixel correction circuit 9, and the defective pixel correction circuit 9 corrects the defective pixel. The output of the defective pixel correction circuit 9 is subjected to digital-analog conversion processing by a digital-analog (DA) converter 10 to be converted into an analog signal, and an image related to the analog signal is displayed on the monitor 11.

  The digital image data that is the output of the AFE circuit 3 is supplied to the motion vector detection circuit 6 and further supplied to the isolated region discrimination circuit 5 via the motion vector detection circuit 6. The isolated area discriminating circuit 5 detects isolated areas existing in the image related to the digital image data and counts the number thereof. More specifically, the isolated area determination circuit 5 detects an isolated area existing in each divided area obtained by dividing the digital image data output from the AFE circuit 3 into a plurality of areas. The motion vector detection circuit 6 obtains an image motion vector based on the digital image data. Here, the isolated region is, for example, a region corresponding to a star in the night sky at the time of shooting the starry sky, and is an island-like region having a pixel value different from that of the surrounding pixels.

  The microcomputer 12 supplies an instruction or the like via the data bus 14 and controls functions (such as the isolated area determination circuit 5 and the memory control circuit 8) in the imaging apparatus. The input key 13 is used by a user who uses the imaging apparatus to perform operations such as input related to various settings and instructions, and outputs a signal to the microcomputer 12 according to the input operation.

  Here, in this embodiment, the motion vector is calculated from images of two frames (or fields) that are temporally continuous. As shown in FIGS. 4A and 4B, the image in the previous field is referred to as a search area 41, and the image in the current field is referred to as a template area 42. The template area 42 has a narrower area than the search area 41. In FIG. 4B, a region 43 corresponding to the search region 41 is indicated by a dotted line for comparison and reference.

  Further, the template area 42 divides the area by a predetermined number. FIG. 4B illustrates an example in which the template region 42 is divided into five in the horizontal direction and four in the vertical direction, that is, five in the horizontal direction and four in the vertical direction. Each divided area is also referred to as a matching block. As shown in FIG. 4C, the correlation is obtained by shifting as described later in each search area 45 obtained by dividing each template area 44 divided as a matching block, and the shift amount that maximizes the correlation value between the two is determined. Let it be a vector.

FIG. 2 is a block diagram showing the internal configuration of the motion vector detection circuit 6.
In FIG. 2, the band pass filter 21 extracts a predetermined frequency component from the digital image data that is the output of the AFE circuit 3. The zero cross detection circuit 22 detects the zero cross point of the output signal of the band pass filter 21 and binarizes the output signal.

  The search memory 23 is a memory that delays the signal corresponding to the search area among the binarized signals that are the output of the zero cross detection circuit 22 by one frame (one field). The template memory 24 is a memory for storing the current frame signal of the binarized signal that is the output of the zero cross detection circuit 22 for the tape rate region. The shift registers 25 and 26 temporarily store the output signals of the search memory 23 and the template memory 24, respectively, and shift them up, down, left and right in the screen.

  An exclusive NOR (EXNOR) integrating circuit 27 performs an exclusive OR operation using the outputs of the shift registers 25 and 26, and integrates the inverted value of the operation result in units of predetermined blocks. The maximum value detection circuit 28 stores the outputs of the EXNOR integrating circuit 27 for each shift by the shift registers 25 and 26, compares them, and detects the maximum value. The maximum value detection circuit 28 outputs, as a motion vector, the shift amount and direction in which the total sum of the outputs of the EXNOR integrating circuit 27 is maximized.

  The search memory 23, template memory 24, shift registers 25 and 26, EXNOR integrating circuit 27, and maximum value detecting circuit 28 described above constitute a matching circuit.

Next, the operation of the motion vector detection circuit 6 will be described.
The band pass filter 21 extracts a predetermined frequency component of the digital image data supplied from the AFE circuit 3, and the zero cross detection circuit 22 binarizes the extracted signal (image data) of the predetermined frequency component. This binarization operation will be described with reference to FIG.

  5A and 5B show the output of the band pass filter 21. FIG. 5A and 5B, the horizontal axis represents coordinates in the horizontal (x) direction of the screen, and the vertical axis represents density (or luminance), which shows a cross section of the two-dimensional space. In FIG. 5A, reference numerals 51, 52, and 53 denote output waveforms of the band pass filter 21 when the same subject is photographed under different illumination conditions.

  The motion vector detection circuit in this embodiment extracts a specific frequency component from the supplied digital image data by the band pass filter 21 and detects a motion vector from the phase information of the specific frequency component. Therefore, the motion vector detection circuit in the present embodiment can obtain a binarized pattern that does not depend on a change in illumination conditions.

  That is, as shown in FIG. 5, in the output waveforms 51, 52, 53 of the band pass filter 21, the zero cross point is constant regardless of the illumination conditions. Since the zero cross point moves according to the movement of the subject, the motion vector can be detected by the change of the zero cross point. Therefore, the zero cross detection circuit 22 detects the zero cross point of the output signal of the band pass filter 21 and binarizes the output of the band pass filter 21 using the zero level obtained by the detection as a threshold value.

  The signal binarized by the zero cross detection circuit 22 is stored in the search memory 23 and the template memory 24 as appropriate. The shift registers 25 and 26 temporarily store the outputs of the search memory 23 and the template memory 24 having a time difference of 1 frame (or 1 field) from each other, and output a signal obtained by shifting the output vertically and horizontally within the screen. The EXNOR integrating circuit 27 outputs the inversion of the exclusive OR of the outputs of the shift registers 25 and 26, and calculates the sum in units of predetermined blocks in the screen. The output value of the EXNOR integrating circuit 27 corresponds to a correlation value in a predetermined block corresponding to each shift amount.

  As described above, the shift registers 25 and 26 shift the image of the previous field and the current field within a predetermined search range, and the EXNOR integrating circuit 27 calculates the inversion of the exclusive logical sum and the total sum for each shift amount. And output to the maximum value detection circuit 28. The maximum value detection circuit 28 compares the outputs of the EXNOR integration circuit 27 with respect to the shift amounts of the shift registers 25 and 26 for each matching block, and determines the shift amount when the output of the EXNOR integration circuit 27 is maximized as a motion vector. Output as. In this way, a motion vector is detected for each matching block obtained by dividing the image.

  With the above operation, a motion vector can be detected for each matching block in one frame (one field) as shown in FIG. From the motion vector for each matching block, an operation such as averaging is performed by the microcomputer 12 shown in FIG. 1 to obtain a final motion vector between frames (fields).

Next, the isolated region discriminating circuit 5 shown in FIG. 1 will be described in detail.
FIG. 3 is a block diagram showing a configuration of the isolated region discrimination circuit 5.
As illustrated in FIG. 3, the isolated region determination circuit 5 includes a test region generation unit 31, an isolated region detection unit 32, a counter 33, and an isolated region size designation unit 34.

  The verification area generation unit 31 generates a verification area corresponding to the isolated area size to be detected designated by the isolated area size designation unit 34 in the matching block from which the binarized data is read from the template memory. The isolated region detection unit 32 performs pattern matching between the pixel value in the verification region and the isolated region pattern for each verification region generated by the verification region generation unit 31, and detects an isolated region. The counter 33 counts the number of isolated areas detected by the isolated area detection unit 32. The isolated region size designation unit 34 controls the verification region generation unit 31 and the isolated region detection unit 32 according to the isolated region size to be detected.

Next, the operation of the isolated region determination circuit 5 will be described with reference to FIGS.
FIG. 6 is a flowchart showing the operation of the isolated region discrimination circuit 5. FIG. 7 is a diagram showing a test area for each isolated area size. FIGS. 8, 9, and 10 are isolated areas whose isolated area sizes are “1”, “2”, and “3”, respectively. FIG.

  When the isolated region determination circuit 5 starts its operation, it reads the data of the matching block (divided template region) from the template memory 24 in the motion vector detection circuit 6 (S101). Next, after the isolated region size is set to “1” (S102), the test region generation unit 31 under the control of the isolated region size designating unit 34 uses the zero cross detection circuit 22 in the motion vector detection circuit 6. A test area is generated based on the binarized data (S103). Specifically, the test area generation unit 31 synchronizes data of necessary areas according to the specified isolated area size.

  For example, when the isolated area size transmitted through the data bus 14 according to the instruction of the microcomputer 12 is “1”, the verification area generation unit 31 starts from 3 × 3 pixels as shown in FIG. Data excluding the pixels at the four corners (pixels marked with “x” in the figure) are synchronized. In FIG. 7A, a black circle (“●”) in the drawing represents a pixel that has been binarized to become black (for example, a pixel in the night sky portion in starry sky photography) (hereinafter, FIG. 7B, The same applies to (c)).

  The five data synchronized by the verification region generation unit 31 are subjected to pattern matching by the isolated region detection unit 32 (S104). As a result, there are four binarized pixels (black circles), and the pixel A1 positioned in the middle is binarized and white (for example, corresponding to a star). In this case, the isolated region detection unit 32 detects the region as an isolated region of size “1”.

  When the isolated area size is “1”, the isolated area shown in FIG. 8A can be detected. Further, when an isolated region of size “1” is detected in the adjacent test region, an isolated region of size “1” adjacent in the oblique direction as shown in FIGS. 8B and 8C can be detected.

  Similarly, when the isolated area size is designated as “2”, the test area generation unit 31 synchronizes data obtained by removing four corners from 4 × 4 pixels as shown in FIG. 7B. The isolated region detection unit 32 has eight pixels (black circles) binarized and black as shown in FIG. 7B, and has four pixels B1, B2, B3, and B4. When an isolated area pattern of size “2” is detected, the area is detected as an isolated area of size “2”. FIGS. 9A to 9E show an isolated region pattern of size “2”. When any of these isolated region patterns is detected, the isolated region detection unit 32 determines the size of the region. It is detected as an isolated region of “2”.

Similarly, when the isolated area size is designated as “3”, the test area generation unit 31 synchronizes data obtained by removing the four corners from 5 × 5 pixels as shown in FIG. . The isolated region detection unit 32 includes all 12 black pixels (black circles) shown in FIG. 7C and detects an isolated region pattern of size “3” using nine pixels C1 to C9. Detects the region as an isolated region of size “3”. FIGS. 10A to 10G show an example of an isolated region pattern having a size “3”. When these isolated region patterns are detected, the isolated region detecting unit 32 converts the region into a size “ 3 ”is detected as an isolated region. Note that FIG. 10 does not show all isolated region patterns of size “3”.
Needless to say, it is possible to detect an isolated region pattern of size “4” or larger by the same method.

  Returning to FIG. 6, as a result of pattern matching in the isolated region detection unit 32, if the pixel value in the verification region matches the isolated region pattern (Y in S105), the counter 33 increments the count value (S106). . On the other hand, if the pixel value in the verification area does not match the isolated area pattern (N in S105), step S106 is skipped.

  Further, if the designated isolated area size does not reach the maximum value (N in S107), the isolated area size is incremented in accordance with an instruction from the microcomputer 12 (S108). The test area generation unit 31 under the control of the isolated area size designation unit 34 generates a test area corresponding to the isolated area size (S103).

  For example, when the detection operation for the isolated region size “1” ends and the operation shifts to the detection operation for the isolated region size “2”, the verification region generation unit 31 sets the verification region to 3 × 3 pixels shown in FIG. From this area, a 4 × 4 pixel test area shown in FIG.

  If the specified isolated area size has reached the maximum value (Y in S107), it is determined whether or not the isolated area detection operation has been performed in all areas in the matching block (S109). As a result, if the detection operation in the entire region has not ended (N in S109), the isolated region determination circuit 5 sets the isolated region size to “1” again (S102), and the detection operation has not ended. The detection operation of a region (for example, a region adjacent to a region that has already been detected) is similarly performed.

  If the result of determination in step S109 is that the isolated region detection operation in all regions within the matching block has ended (Y in S109), the motion vector detection circuit 5 looks at the count value of the counter 33. If the count value is equal to or greater than the predetermined value, that is, if a certain number or more of isolated regions exist in the matching block (Y in S110), the motion vector detection circuit 5 performs the motion of the image of the matching block by the above-described operation. A vector is calculated (S111). On the other hand, if there are no more than the specified number of isolated regions in the matching block (N in S110), step S111 is skipped.

  Then, it is determined whether or not the isolated region detection operation has been completed for all the matching blocks in the template region. If completed (Y in S112), the operation is terminated. On the other hand, if the detection operation in all the matching blocks is not completed (N in S112), the count value of the counter 33 is reset (S113), the data of the next matching block is read (S114), and the above-described isolated region Detection operation is performed.

An example of the isolated area and the template area detected by the above-described operation is shown in FIG.
In the example shown in FIG. 11, five matching blocks are arranged in the horizontal direction and four matching blocks in the vertical direction. In FIG. 11, a white isolated region corresponds to, for example, a star at the time of shooting a starry sky. When the above-mentioned default value for calculating the motion vector is “2”, areas where two or more isolated areas exist (areas surrounded by bold lines) 0-A, 0-D, 1-A, 1-E, A motion vector is calculated by 2-B, 2-D, and 3-E. In the matching block that does not calculate the motion vector, control is performed based on the count value of the counter 33 shown in FIG. 3, and the shift registers 25 and 26, the EXNOR integrating circuit 27, and the maximum value detecting circuit 28 shown in FIG. It suffices to stop the operation. By doing so, it is possible to prevent unnecessary motion vector calculation processing from being performed, and to reduce processing load and power consumption.

  Based on the motion vector obtained as described above, the cutout position (readout range) is controlled by the memory control circuit 8 according to the instruction of the microcomputer 12, and the video signal one frame (or one field) before stored in the memory 7. Electronic vibration isolation is performed by reading

  As described above, a motion vector is not calculated for a matching block in which the number of isolated regions existing is less than a predetermined number (0 or 1 in the present embodiment), thereby avoiding the conventional inconvenience that occurs during starry sky shooting. Even at times, the motion vector can be calculated accurately. As a result, accurate electronic image stabilization can be performed even when shooting a starry sky, and a good captured image can be obtained.

  In the above-described isolated region detection operation, the specified isolated region size is incremented, and the isolated region is sequentially detected with the isolated region size of 1 to the maximum value. However, the present invention is not limited to this. It is not a thing. For example, the isolated area size may be first set to the maximum value, and the isolated area size may be decremented to detect the isolated area with the isolated area size of 1 to the maximum value. It suffices if the isolated region can be detected with all the isolated region sizes of 1 to the maximum value, and the designation order of the isolated region size may be random or arbitrary.

Next, the defective pixel correction in the defective pixel correction circuit 9 in the present embodiment will be described.
As in the case of realizing electronic image stabilization, in order to correct a defective pixel, the cutout position is controlled by the memory control circuit 8 based on the motion vector obtained as described above, and stored in the memory 7. This is done by reading the video signal of one frame (or one field) before.

  For example, when the obtained motion vectors are (vec_h, vec_v) in the horizontal direction and the vertical direction, respectively, the video signal of one frame (or one field) is the output of the memory 7 and the video signal of the current field Is the output of the camera signal processing circuit 4. Therefore, as shown in FIG. 12, if the signal of the image 122 at a location shifted by (vec_h, vec_v) in the TV signal from the image 121 based on the output of the memory 7 is compared, the two should match.

  However, as shown in FIG. 12, when the isolated region 123 exists only in the image 121 and the corresponding isolated region 124 does not exist in the image 122, the region 123 in the image 121 is a defective pixel (white) in the image sensor 2. It can be determined that it is a scratch. Therefore, the defective pixel correction circuit 9 performs correction to remove the region 123 that is the detected defective pixel.

  The removal of the defective pixel area by the defective pixel correction circuit 9 may be, for example, simply a black pixel or interpolation with peripheral pixels of the area 123, when shooting at the starry sky. The pixel value of the corresponding region 124 in the image 122 may be used. Moreover, it is not limited to these examples, You may make it apply the correction method used conventionally.

  As described above, the defective pixel correction circuit compares the area of the previous frame with the area of the current frame moved by the motion vector from the area corresponding to the area, and erases the isolated area existing in only one of the areas. In step 9, correction is performed. Thereby, at the time of photographing the starry sky, while performing the image stabilization operation, the portion (pixel) corresponding to the star in the captured image is distinguished from the defective pixel (white defect) in the image sensor 2, and only the defective pixel (white defect) is corrected. Can do.

  In the above-described embodiment, the case where the motion vector is calculated only for the matching block having two or more isolated regions has been described as an example. However, the predetermined number of isolated regions (predetermined value) for calculating the motion vector is not limited to “2”, and the same effect can be obtained even if it is “3” or more.

  In addition, the method for detecting an isolated region described in the present embodiment is not limited to this, and other methods may be used.

  In the present embodiment, in a matching block that does not calculate a motion vector, control is performed based on the count value of the counter 33, and the operations of the shift registers 25 and 26, the EXNOR integrating circuit 27, and the maximum value detecting circuit 28 are stopped. showed that. However, the present invention is not limited to this. For example, when motion vectors are obtained for all the matching blocks and the final motion vector is obtained by the microcomputer 12, the motion of the matching block having an insufficient number of isolated regions. A vector may not be used.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.
The image pickup apparatus according to the second embodiment performs motion when the number of matching blocks in which an isolated region of a predetermined number or more exists is extremely small and the motion vector cannot be calculated or the accuracy of the calculated motion vector is poor. The vector can be calculated accurately. Specifically, in the second embodiment, when the number of matching blocks in which a certain number or more of isolated regions exist is extremely small, the binarization pattern is changed by controlling the pass band of the band pass filter 21. Thus, the motion vector can be calculated.

  The configuration of the imaging apparatus according to the second embodiment is the same as that of the imaging apparatus according to the first embodiment shown in FIGS.

The operation of the imaging apparatus according to the second embodiment will be described with reference to FIG.
FIG. 13 is a flowchart showing the operation in the second embodiment.
When the operation is started, the microcomputer 12 sets a pass band of the band pass filter 21 as a default via the data bus 14 (S201). Next, data of the first matching block is read from the template memory 24 in the motion vector detection circuit 6 (S202). Subsequently, the number of isolated regions present in the matching block is counted in the same manner as in the first embodiment described above (S203).

  If it is determined that there are more than a predetermined number of isolated areas in the matching block (Y in S204), it is determined that this matching block is valid, and an effective template area counter (not shown) is incremented (S205). On the other hand, when it is determined that the number of isolated regions in the matching block is less than the predetermined number (N in S204), step S205 is skipped.

  Next, it is determined whether or not the isolated region detection operation has been completed for all the matching blocks. If not (N in S206), the data of the next matching block is read (S207), and the process returns to step S203. .

  On the other hand, if the detection of isolated regions has been completed for all matching blocks (Y in S206), the value of the effective template region counter is read, and a predetermined number of effective matching blocks having a predetermined number or more of isolated regions are included in one frame. It is determined whether or not there is more (S208). As a result, if the number of valid matching blocks is equal to or larger than the predetermined number (Y in S208), the count value of the valid template area counter is reset (S210), the operation is terminated, and the motion vector described in the first embodiment is continued. Detection and defective pixel correction are performed.

  On the other hand, if the result of determination in step S208 is that there are not more than a predetermined number of valid matching blocks (N in S208), the microcomputer 12 changes the passband of the bandpass filter 21 (S209). Here, for example, it is assumed that the isolated region is first detected using the band-pass filter 21 having a pass band from which an output waveform as shown in FIG. At this time, since the interval between the zero cross points is wide as shown in FIG. 5A, at the time of shooting the starry sky, there is a high possibility that image data corresponding to a small star will not become an isolated point when binarized. .

  Therefore, when the pass band of the band pass filter 21 is increased, the binarized signal becomes as shown in FIG. 5B, and the interval between the zero cross points is narrowed, and a smaller star can be generated as an isolated region. it can. Further, at the time of shooting the starry sky, most subjects are only the night sky and the stars, and the change of the zero cross point due to the change of the illumination condition as described in the first embodiment cannot occur. Therefore, when an extremely small star is desired to be an isolated region, the band pass filter 21 may be controlled to stop the operation, and a binarized signal may be generated using a certain threshold value.

  As described above, when the number of matching blocks having a predetermined number or more of isolated regions is smaller than a predetermined number, the size of the star is increased by changing the passband of the bandpass filter 21 included in the motion vector detection circuit 5. It is possible to realize the image stabilization operation even when the height is small. Further, by performing the same motion vector detection and defective pixel correction operation as in the first embodiment, the same effects as in the first embodiment can be obtained.

(Other embodiments of the present invention)
For realizing the functions of the above-described embodiment for a computer (CPU or MPU) in an apparatus or system connected to the various devices so as to operate various devices to realize the functions of the above-described embodiments. Supply software programs. And what was implemented by operating the said various devices according to the program stored in the computer of the system or apparatus is also contained under the category of this invention.
In this case, the software program itself realizes the functions of the above-described embodiments, and the program itself constitutes the present invention. Further, means for supplying the program to the computer, for example, a recording medium storing the program constitutes the present invention. As a recording medium for storing such a program, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.
In addition, such a program is also included in the embodiment of the present invention when the function of the above-described embodiment is realized in cooperation with an operating system running on a computer or other application software. Needless to say.
Further, after the supplied program is stored in a memory provided in a function expansion board or a function expansion unit related to the computer, a CPU or the like provided in the function expansion board or the like based on an instruction of the program may be part of the actual processing Do everything. Needless to say, the present invention includes the case where the functions of the above-described embodiments are realized by the processing.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

It is a figure which shows the structural example of the imaging device by the 1st Embodiment of this invention. It is a figure which shows the structural example of the motion vector detection circuit in 1st Embodiment. It is a figure which shows the structural example of the isolated area | region discrimination circuit in 1st Embodiment. It is a figure which shows the search area | region and template area | region which concern on a motion vector detection. It is a figure which shows the output waveform of the bandpass filter of a motion vector detection circuit. 5 is a flowchart illustrating an operation of the isolated region determination circuit according to the first embodiment. It is a figure which shows the test area | region with respect to each isolated area size. It is a figure which shows the shape of the isolated area whose isolated area size is "1". It is a figure which shows the shape of the isolated area whose isolated area size is "2". It is a figure which shows the shape of the isolated area whose isolated area size is "3". It is a figure which shows the isolated area | region detected with the template area | region. It is a figure for demonstrating the defective pixel correction | amendment in 1st Embodiment. It is a flowchart which shows operation | movement of the imaging device by 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens 2 Image pick-up element 4 Camera signal processing circuit 5 Isolated area discrimination circuit 6 Motion vector detection circuit 7 Memory 8 Memory control circuit 9 Defective pixel correction circuit 12 Microcomputer 14 Data bus 21 Band pass filter 22 Zero cross detection circuit 23 Search memory 24 Template Memory 25, 26 Shift register 27 EXNOR circuit 28 Maximum value detection circuit 31 Test region generation unit 32 Isolated region detection unit 33 Counter 34 Isolated region size designation unit

Claims (11)

An imaging device having a plurality of pixels;
The first image data output from the image sensor is divided into a plurality of areas, and the motion vector associated with each divided area is obtained by referring to the second image data output prior to the first image data. A motion vector detecting means for calculating;
An isolated area discriminating means for discriminating an isolated area existing in the divided area of the first image data for each divided area;
The imaging apparatus according to claim 1, wherein the motion vector detection unit calculates a motion vector related to the divided region in accordance with a determination result of the isolated region determination unit.   The image processing apparatus further comprises image stabilization processing means for performing image stabilization processing by controlling a read range of the image data output from the image sensor based on the motion vector calculated by the motion vector detection means. The imaging apparatus according to 1.   The imaging apparatus according to claim 1, wherein the motion vector detection unit calculates a motion vector related to a divided region determined by the isolated region determination unit that a predetermined number or more of isolated regions exist.   The imaging device according to any one of claims 1 to 3, wherein the first image data and the second image data are image data of two frames that are temporally continuous. . The isolated region discriminating means
In the divided area, a test area generating means for generating a test area according to a specified isolated area size;
An isolated area detecting means for detecting an isolated area by performing pattern matching between a pixel value of the verification area generated by the verification area generating means and an isolated area pattern prepared in advance;
The imaging apparatus according to claim 1, further comprising a counting unit that counts the number of isolated regions detected by the isolated region detecting unit.   6. The image pickup apparatus according to claim 5, wherein the isolated area size is sequentially changed, and an isolated area existing in the divided area is detected for all sizes of the specifiable isolated area size.   Correcting means for comparing the first image data with the second image data, correcting an isolated area existing in only one of the first image data and the second image data, and erasing the isolated area; The imaging apparatus according to claim 1, further comprising: The motion vector detection means has a filter means for extracting a specific frequency component of image data from the image sensor,
8. The frequency component extracted by the filter means is changed when the number of divided regions having an isolated region equal to or larger than the predetermined number is smaller than the predetermined number. Imaging device. First image data output from an imaging device having a plurality of pixels is divided into a plurality of areas, and each divided area is referred to with reference to the second image data output before the first image data. A motion vector detecting step for calculating the motion vector,
An isolated region determining step for determining, for each divided region, an isolated region present in the divided region of the first image data,
An imaging method comprising: calculating a motion vector relating to the divided region in the motion vector detecting step in accordance with a discrimination result in the isolated region discriminating step. First image data output from an imaging device having a plurality of pixels is divided into a plurality of areas, and each divided area is referred to with reference to the second image data output before the first image data. A motion vector detection step for calculating the motion vector,
Causing the computer to execute an isolated region determination step for determining, for each divided region, an isolated region existing in the divided region of the first image data,
In the motion vector detection step, a motion vector related to the divided region is calculated according to the determination result in the isolated region determination step.   A computer-readable recording medium on which the program according to claim 10 is recorded.

Images