JP2008206111A - Photographing apparatus and photographing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a photographing apparatus capable of excellent image data for both an object and a background even on an illumination condition of backlighting or excessive frontlighting, etc. <P>SOLUTION: An object is detected from a frame F(i) and a luminance histogram of an object region is created. If there is not 20% or more of luminance distribution in an intermediate luminance range, under exposure control for making proper the luminance distribution in the object region and a background region (S13, S15-S17), frames F(i+1) and F(i+2) are obtained, respectively (S14, S18). Spatial frequency analysis is performed on the object region and the background region (by luminance ranges) in the three frames (S12, S19, S20), results thereof are compared, image data of the frames F(i+1), F(i+2) are weighted more greatly as an exposure region becomes proper (S21-S23), and a frame Fcom combining the post-processing frames F(i+1), F(i+2) is then outputted. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

The present invention relates to a photographing apparatus and a photographing method, and relates to an image processing technique for outputting an image of a subject to be noticed and a background that are easy to see even under illumination conditions that result in a high contrast state such as backlight or excessive forward light.

Conventionally, in surveillance systems, shooting is often performed with attention paid to a subject such as a person or a car. However, the shooting environment for the subject is not always in good lighting conditions. For example, in high-contrast scenes represented by over-adjacent light outdoors and backlighting when shooting outside the building, the shutter speed, aperture, and gain are adjusted to ensure proper exposure of the entire image. Even in this case, the subject or background may be photographed with a whiteout or blackout state.

To prevent these phenomena, automatic exposure control (Automatic Exposure) divides the field of view into a plurality of areas such as rectangles, and determines the overall exposure by weighting the amount of light detected for each divided area. : AE) and an adjustment function such as a backlight correction function (Back Light Control: BLC) for performing exposure control in accordance with a pre-set attention area around the entire screen or near the center of the screen. If the subject portion becomes too dark or too bright, or there is a problem that proper photometry cannot be performed on a subject that appears and moves at an arbitrary position, and a sufficient effect may not be obtained. On the other hand, if the exposure is adjusted only for the subject, the background may be overexposed or darkened and may not be visible.

On the other hand, if an image sensor with a wide dynamic range is used, it can be adapted to illumination conditions with high contrast and the SN ratio is improved, but the image sensor is naturally expensive. Accordingly, proposals have been made from various angles for obtaining a high-quality image even in the case of the illumination condition without depending on the performance of the image sensor.

For example, in Patent Document 1 below, two image data captured at two different exposure times of short exposure and long exposure are divided into a plurality of blocks, and the luminance system frequency component parameter and the luminance system luminance level component obtained for each block are divided. The image composition parameter for each block is obtained using the parameter and the color system component parameter, and the image composition parameter of each block corresponding to each image data and a predetermined threshold value (threshold value for determining whether proper exposure or abnormal exposure) And a block corresponding to the closer image composition parameter is selected for composition and is used as one image data. Also, in Patent Document 2 below, a frame image is captured in a cycle shorter than the output cycle of the output image, the captured frame image is subjected to frequency analysis to identify a low contrast region, and the image of the identified region An image processing apparatus that creates an output image by adding data for a plurality of frames is disclosed.
JP 2001-8216 A Japanese Patent Application Laid-Open No. 2003-259211

By the way, the following problems are pointed out also in the image processing apparatuses disclosed in the patent documents. First, in the image processing apparatus of Patent Document 1, image data captured with two fixed exposure times, long and short, is used, but any exposure time may be far from proper exposure. In addition, a more appropriate block is selected as a synthesis target by comparing the image synthesis parameter with a predetermined threshold value. Of course, blocks having different exposure times are often adjacent to each other and are individual elements of the image synthesis parameter. Since each component parameter is also obtained as an average value for each pixel in the block, a case where a bright and dark seam is conspicuous at the boundary of the block is assumed. Furthermore, since the subject itself may inherently have high and low brightness, the result of comparing the image synthesis parameter and the predetermined threshold as described above does not always result in proper selection of the block. There is also a problem of not.

In the image processing apparatus of Patent Document 2, a decrease in contrast of a region of interest in an image is likely to occur in an overexposure condition or in a blackout condition due to insufficient illumination due to a backlight condition. Since information such as the original hue and contour is often missing, it is not always possible to effectively reproduce image information even if a plurality of such frames are added and combined. In addition, there are many cases where noise is mixed in a dark portion in an image, and the addition / combination of a plurality of frames may improve the contrast, but an improvement in the S / N ratio is not desired. There is a possibility that a contrast image in which an erroneous part is emphasized may be output, so there is a question as to whether or not it leads to an improvement in image quality.

Therefore, the present invention solves the problems in the prior art, and obtains a higher quality image even under illumination conditions in which the entire image is in a high contrast state such as backlight or excessive forward light. An object of the present invention is to provide a photographing apparatus and a photographing method capable of performing the above.

According to a first aspect of the present invention, in a photographing apparatus having an automatic exposure control function, a first frame storage unit that stores first frame data of a captured image and a first frame storage unit that stores the first frame data. Subject detection means for detecting a target subject from first frame data; first histogram creation means for creating a luminance histogram of the subject area when the subject detection means detects a subject; Determining means for determining whether or not the distribution of the cumulative number of pixels in the luminance histogram of the subject area created by the histogram creating means is included in a predetermined ratio or more in an intermediate luminance range set in advance as a luminance range regarded as a proper exposure state And the first frame stored in the first frame storage means when the determination result of the determination means is negative. A second histogram creating unit that creates a luminance histogram of a region other than the subject region (hereinafter referred to as a “background region”) in the data and the determination result of the determination unit is negative. A spatial frequency analysis for the subject region in the first frame data stored by the frame storage means and a spatial frequency analysis for each region belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region are performed. When the determination result of the first spatial frequency analysis means and the determination means is negative, the distribution of the cumulative number of pixels in the luminance histogram of the subject area created by the first histogram creation means is approximately the intermediate First exposure control means for performing exposure control so as to be within a luminance range, and imaging in an exposure state set by the first exposure control means Distribution of the cumulative number of pixels in the luminance histogram of the background region created by the second frame storage means for storing the second frame data and the second histogram creation means is unevenly distributed in the high luminance range or the low luminance range A second exposure control means for performing exposure control so as to expand to the intermediate luminance range side from the obtained state, and a third frame data for storing the third frame data imaged in the exposure state set by the second exposure control means. 3 frame storage means, spatial frequency analysis of the subject area in the second frame data stored by the second frame storage means, and the intermediate brightness range, high brightness range, and low brightness range of the background area A second spatial frequency analyzing means for executing a spatial frequency analysis for each region, and a third frame stored by the third frame storage means. Third spatial frequency analysis means for performing a spatial frequency analysis for a subject area in frame data and a spatial frequency analysis for each area belonging to the intermediate luminance range, the high luminance range, and the low luminance range of the background region; Each analysis result by the second spatial frequency analysis means and the third spatial frequency analysis means with respect to the analysis result by the first spatial frequency analysis means is respectively compared, and the second and third frame storage means respectively A weighting unit that applies a larger weighting process to the second and third frame data stored in each of the regions in a more appropriate exposure state, and after the weighting process is performed by the weighting unit The second frame data of the second frame storage means and the third frame data of the third frame storage means. Frame synthesizing means for synthesizing the frame data, and when the subject detecting means detects the subject and the judgment result of the judging means is negative, outputs the frame synthesized by the frame synthesizing means. In other cases, the present invention relates to an imaging apparatus that outputs a frame by performing automatic exposure control that sets an exposure state based on the light amount of the entire frame.

According to the first aspect of the present invention, the image is taken under an illumination condition that results in a high contrast state such as backlight or over-order light, and is determined by the determination unit as “from the first frame data generated by the first histogram generation unit. First, when it is determined that the distribution of the cumulative number of pixels in the luminance histogram of the subject area is not included within the intermediate luminance range by a predetermined ratio or more (when the subject area is not in an appropriate exposure state) The exposure state for optimizing the luminance distribution of the subject area is set by the first exposure control means to obtain the second frame data. For the background area, the second exposure control means refers to the brightness histogram of the background area, so that the distribution of the cumulative number of pixels is spread from the state of being unevenly distributed in the high brightness range or the low brightness range to the intermediate brightness range side. Thus, the exposure state is set to obtain the third frame data. Therefore, it seems that if the subject area of the second frame data and the background area of the third frame data are combined, an image with good contrast without overexposure and underexposure can be obtained for the entire frame. Each of the second and third frame data is obtained by exposure control based on the luminance histogram, and a high-quality image cannot be obtained even if the subject area and the background area are synthesized as they are. Therefore, according to the present invention, not only the luminance histogram of the subject area is created by the first histogram creating means for the first frame data, but also the subject area and the background area (each luminance range) by the first spatial frequency analyzing means. The second and third frame data are also subjected to spatial frequency analysis by the second and third spatial frequency analysis means, respectively. Then, the weighting unit compares the analysis result by the first spatial frequency analysis unit and the analysis result by the second and third spatial frequency analysis units for each region, and the second and third frame data are compared. On the other hand, as the exposure state is more appropriate for each area, a larger weight is given, and the frame synthesizing unit synthesizes the second and third frame data after the weighting process. By performing weighting according to the appropriate degree of exposure based on the result of the spatial frequency analysis, it is possible to display a region in the image that may be overexposed or underexposed with good contrast.

According to a second aspect of the present invention, in a photographing apparatus having an automatic exposure control function, a first frame storage unit that stores first frame data of a captured image, and a first frame stored by the frame storage unit Subject detection means for detecting a target subject from frame data; first histogram creation means for creating a luminance histogram of the subject area when the subject detection means detects a subject; and creation of the first histogram Determining means for determining whether the distribution of the cumulative number of pixels in the luminance histogram of the subject region created by the means is included in a predetermined ratio or more in an intermediate luminance range set in advance as a luminance range regarded as a proper exposure state; The first frame data stored in the first frame storage means when the determination result of the determination means is negative A second histogram creating means for creating a luminance histogram of the background region in the first area, and a subject in the first frame data stored in the first frame storage means when the determination result of the determination means is negative First spatial frequency analysis means for performing spatial frequency analysis for a region, and spatial frequency analysis for each region belonging to the intermediate luminance range, high luminance range, and low luminance range of a background region; and When the determination result is negative, the first exposure control is performed so that the distribution of the cumulative number of pixels in the luminance histogram of the subject area created by the first histogram creating means is substantially within the intermediate luminance range. An exposure control means and a second frame for storing the second frame data imaged in the exposure state set by the first exposure control means Frame storage means, third histogram creation means for creating a luminance histogram of the subject area in the second frame data stored by the second frame storage means, and background area created by the second histogram creation means Second exposure control means for performing exposure control so that the distribution of the cumulative number of pixels in the brightness histogram of the brightness histogram spreads toward the intermediate brightness range from a state of being unevenly distributed in the high brightness range or the low brightness range; and the second exposure control means Creates a third frame storage means for storing the third frame data captured in the exposure state set by the camera, and a luminance histogram of the background area in the third frame data stored by the third frame storage means. The fourth histogram generating means and the second frame data stored in the second frame storing means. Second spatial frequency analysis means for executing a spatial frequency analysis for a subject area and a spatial frequency analysis for each area belonging to the intermediate luminance range, the high luminance range, and the low luminance range of the background region, Spatial frequency analysis for the subject area in the third frame data stored by the third frame storage means, and spatial frequency analysis for each area belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region. The luminance histogram of the subject area in the first frame data created by the third spatial frequency analyzing means, the first histogram creating means, and the second frame created by the third histogram creating means, respectively. Cumulative number of pixels in the luminance histogram by comparing with the luminance histogram of the subject area in the data The first comparison / determination means for determining whether or not the distribution has shifted from the low luminance range to the intermediate luminance range by a predetermined amount or more, and the determination result of the first comparison / determination means is negative In addition, the analysis result of the subject area in the first frame data by the first spatial frequency analysis means is compared with the analysis result of the subject area in the second frame data by the second spatial frequency analysis means. Then, when the determination result by the second comparison / determination means for determining whether or not the noise in the subject area has increased to a predetermined value or more and the determination result by the second comparison / determination means are affirmative, The amount of noise included in each subject area in the first to third frame data respectively stored in the first to third frame storage means is detected, and each subject area is detected. A first weighting unit that applies a smaller weighting process to the image data as the noise amount increases, a luminance histogram of a background region in the first frame data created by the second histogram creating unit, and the fourth Is compared with the luminance histogram of the background area in the third frame data created by the histogram creating means, and whether the distribution of the cumulative number of pixels of the luminance histogram has shifted from the low luminance range to the intermediate luminance range by a predetermined amount or more If the determination result of the third comparison / determination means and the third comparison / determination means is negative in the first frame data by the first spatial frequency analysis means Analysis of each region belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region (hereinafter referred to as “each target region”) The result is compared with the analysis result for each target area of the background area in the third frame data by the third spatial frequency analyzing means, and the noise is increased to a predetermined value or more in each target area. The fourth comparison / determination means for determining whether or not there is a target area in which noise has increased to a predetermined value or more by the fourth comparison / determination means; The amount of noise included in the corresponding target area of the first to third frame data respectively stored by each of the frame storage means is detected, and the larger the noise amount with respect to the image data of the corresponding target area, Of the analysis results obtained by the second weighting means for performing smaller weighting processing and the first to third spatial frequency analysis means, the first weighting means and the second weighting means For each region of the first to third frame data stored in the first to third frame storage means, respectively. A third weighting unit that applies a larger weighting process to a region in a more appropriate exposure state and the first to third frame storage units that have been subjected to the weighting process by the third weighting unit respectively store Frame synthesizing means for synthesizing the first to third frame data, and when the subject detecting means detects the subject and the judgment result of the judging means is negative, The frame synthesized by the frame synthesizing unit is output. In other cases, the frame is obtained by performing automatic exposure control for setting the exposure state based on the light amount of the entire frame. According to imaging apparatus which comprises causing output.

If an object that originally has low brightness exists in the captured image, the first invention processes it without distinguishing it from the case of low brightness depending on the illumination conditions. If you try to set the state, it will be overexposed. In other words, in an area where the brightness of the object is low, the brightness does not increase even when the exposure is increased, and only the noise increases due to overexposure. The low brightness area becomes rough and the image quality of the display image deteriorates. According to the second aspect of the present invention, there is provided an imaging apparatus capable of suppressing the influence of noise correspondingly even when an object with originally low brightness exists in a captured image. In the present invention, the first comparison / determination unit, the second comparison / determination unit, and the first weighting unit are involved with respect to the subject area, and the third comparison / determination unit and the fourth comparison unit are associated with the background area. Comparison determination means and second weighting means are involved. That is, in the subject area and each target area in the first to third frame data, the luminance does not increase even when the luminance distribution is optimized, and the amount of noise increases by a predetermined value or more in the spatial frequency analysis result. The region is considered to be a region having low brightness as an object attribute, and smaller weighting processing is performed on the image data included in the region as the noise amount increases. Further, according to the present invention, the third weighting means compares the results of the spatial frequency analysis in the first to third frame data other than the areas to be subjected to the weighting process, thereby obtaining a more appropriate exposure state. A larger weighting process is applied to a certain area. Then, the first to third frame data after the respective weighting processes are performed by the frame synthesizing unit is synthesized, but the weighting for the region where the luminance is low as the object attribute as described above is reduced. Therefore, a high-quality image in which the influence of noise is suppressed can be obtained.

The first and second inventions can also be understood as inventions for the following photographing methods.
(1) A photographing method using a photographing apparatus having an automatic exposure control function, the first step of storing first frame data of a captured image in a first frame storage means, and the first step A second step of detecting a target subject from the first frame data stored in the frame storage means, and when a subject is detected in the second step, a luminance histogram of the subject region is created The distribution of the cumulative number of pixels in the luminance histogram of the subject area created in the third step and the third step is included in the intermediate luminance range set in advance as the luminance range regarded as the appropriate exposure state at a predetermined ratio or more. A fourth step for determining whether or not the first frame storage means stores the result when the determination result in the fourth step is negative. When the determination result in the fifth step of creating the luminance histogram of the background region in the first frame data and the fourth step is negative, the first frame storage means stores Performing a spatial frequency analysis on the subject area in the first frame data, and a spatial frequency analysis on each of the areas belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region, respectively. If the determination results in the step and the fourth step are negative, the distribution of the cumulative number of pixels in the luminance histogram of the subject area created in the third step is substantially within the intermediate luminance range. In this way, the second frame data captured in the exposure state set in the seventh step for performing exposure control and the sixth step is used as the second frame data. The intermediate luminance range from the state in which the distribution of the cumulative number of pixels in the luminance histogram of the background region created in the eighth step and the fifth step is unevenly distributed in the high luminance range or the low luminance range. A ninth step of performing exposure control so as to expand to the side, and a tenth step of storing the third frame data captured in the exposure state set in the ninth step in the third frame storage means, , The spatial frequency analysis of the subject area in the second frame data stored in the second frame storage means in the eighth step, and the intermediate luminance range, high luminance range, and low luminance range of the background region. In the eleventh step of performing spatial frequency analysis for each region to which each belongs, and in the tenth step, the third frame storage means A spatial frequency analysis for the subject region in the remembered third frame data and a spatial frequency analysis for each region belonging to the intermediate luminance range, the high luminance range, and the low luminance range of the background region are respectively performed. And the analysis results in the sixth step are compared with the analysis results in the eleventh step and the twelfth step, respectively, and the second and third frame storage means respectively store them. The third and third frame data are subjected to weighting processing in the thirteenth step and the thirteenth step in which a larger weighting process is performed in a region in a more appropriate exposure state. The second frame data of the second frame storage means and the third frame data of the third frame storage means after 14th step of combining, and when the subject is detected in the second step and the determination result in the fourth step is negative, the object is combined in the 14th step. An imaging method, wherein a frame is output, and in other cases, the frame is output by performing automatic exposure control for setting an exposure state based on the light amount of the entire frame.
(2) A photographing method using a photographing apparatus having an automatic exposure control function, wherein a first step of storing first frame data of a captured image in a first frame storage means; A second step of detecting a target subject from the first frame data stored in the frame storage means, and when a subject is detected in the second step, a luminance histogram of the subject region is created The distribution of the cumulative number of pixels in the luminance histogram of the subject area created in the third step and the third step is included in the intermediate luminance range set in advance as the luminance range regarded as the appropriate exposure state at a predetermined ratio or more. A fourth step for determining whether or not the first frame storage means stores the result when the determination result in the fourth step is negative. When the determination result in the fifth step of creating the luminance histogram of the background region in the first frame data and the fourth step is negative, the first frame storage means stores Performing a spatial frequency analysis on the subject area in the first frame data, and a spatial frequency analysis on each of the areas belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region, respectively. If the determination results in the step and the fourth step are negative, the distribution of the cumulative number of pixels in the luminance histogram of the subject area created in the third step is substantially within the intermediate luminance range. In this way, the second frame data captured in the exposure state set in the seventh step for performing exposure control and the sixth step is used as the second frame data. An eighth step of storing the image data in the frame storage means, a ninth step of creating a luminance histogram of the subject area in the second frame data stored in the second frame storage means, and the fifth step A tenth step of performing exposure control so that the distribution of the cumulative number of pixels in the luminance histogram of the background region created in the step spreads from the state unevenly distributed in the high luminance range or the low luminance range to the intermediate luminance range side; An eleventh step for storing the third frame data captured in the exposure state set in step 10 in the third frame storage means; and a third step stored in the third frame storage means. A twelfth step of creating a luminance histogram of the background area in the frame data, and a second number stored in the second frame storage means A thirteenth step of executing a spatial frequency analysis for the subject region in the frame data of the first frame and a spatial frequency analysis for each region belonging to the intermediate luminance range, the high luminance range, and the low luminance range of the background region; Spatial frequency analysis for the subject area in the third frame data stored in the third frame storage means, and spatial frequency analysis for each area belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region The luminance histogram of the subject area in the first frame data created in the third step and the subject in the second frame data created in the ninth step Compared to the luminance histogram of the area, the distribution of the cumulative number of pixels in the luminance histogram is low. A fifteenth step of determining whether or not a predetermined amount or more has shifted from the degree range to the intermediate luminance range, and a determination result in the fifteenth step is negative in the sixth step Comparing the analysis result of the subject area in the first frame data with the analysis result of the subject area in the second frame data in the thirteenth step, and the noise in the subject area exceeds a predetermined value The sixteenth step for determining whether or not there is an increase, and when the determination result in the sixteenth step is affirmative, the first to third frame storage means respectively store the The amount of noise included in each subject area in the first to third frame data is detected, and the smaller the amount of noise relative to the image data of each subject area, the smaller the amount. The luminance histogram of the background area in the first frame data created in the fifth step and the luminance histogram of the background area created in the twelfth step In the eighteenth step of determining whether or not the distribution of the cumulative number of pixels in the luminance histogram has shifted from the low luminance range to the intermediate luminance range by a predetermined amount or more, the determination result in the eighteenth step is negative. Each region belonging to the intermediate luminance range, high luminance range and low luminance range of the background region in the first frame data in the sixth step (hereinafter referred to as “each target region”). And the analysis result for each target region of the background region in the third frame data in the fourteenth step, respectively, A nineteenth step of determining whether or not there is a target area where noise has increased to a predetermined value or more; and in the nineteenth step, it is determined that there is a target area where noise has increased to a predetermined value or more In this case, the amount of noise included in the corresponding target area of the first to third frame data respectively stored in the first to third frame storage means is detected, and the image data of the corresponding target area is detected. Among the analysis results of each frequency analysis in the twentieth step of applying a smaller weighting process as the noise amount is larger, the sixth step, the thirteenth step, and the fourteenth step, The first to third frame storage units store the analysis results for the areas that are not subject to the weighting process in the seventeenth step and the twentieth step, respectively. In each of the first to third frame data areas, a 21st step of applying a larger weighting process to an area in a more appropriate exposure state, and a weighting process in the 21st step. And the 22nd step of synthesizing the 1st to 3rd frame data respectively stored in the first to 3rd frame storage means, and the subject is detected in the 2nd step. If the determination result in the fourth step is negative, the frame synthesized in the twenty-second step is output; otherwise, exposure is performed based on the light amount of the entire frame. An image-taking method comprising: performing automatic exposure control for setting a state and outputting a frame.

The photographing apparatus and the photographing method of the present invention have the above-described configuration, thereby providing the following effects. Even under lighting conditions that result in high contrast, such as in backlighting conditions or in the order of light, it is possible to produce a high-resolution shot image with no blackout or overexposure in either the subject area or background area of the image frame. Make it possible to get. Also, if the object itself has low brightness as its attribute, optimizing the exposure state will cause overexposure and noise in the low brightness area, but it will be output. High image quality is achieved by preventing the effect from appearing on the image. Therefore, the present invention provides a photographing apparatus and a photographing method suitable for the monitoring system.

Hereinafter, embodiments of a photographing apparatus and a photographing method of the present invention will be described with reference to the drawings.
[Embodiment 1]
First, FIG. 1 is a block diagram of a surveillance camera, 11 is an imaging optical system unit including an objective lens and a diaphragm / focus adjustment mechanism, and 12 is a CCD (Charge Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductor) sensor. The configured image sensor, 13 is an AFE (Analogue Front End) that performs analog signal processing such as noise removal [CDS (Correllated Double Sampling) circuit] and gain adjustment [AGC (Auto Gain Control) circuit], etc., 14 An A / D converter that converts the analog signal processed by the AFE 13 into a digital signal, 15 is a memory, 16 is an analysis of frame data saved in the memory 15 to create focus / exposure control data and various data processing A digital signal processing unit (DSP) 17 performs a D / A conversion for converting a digital signal of image data processed by the DSP 16 into an analog signal. , 18 is a lens / aperture drive unit that drives the aperture / focus adjustment mechanism of the imaging optical system unit 11 based on the focus / exposure control data created by the DSP 16, and 19 is based on the focus / exposure control data created by the DSP 16. A timing generator for generating pulses for setting the shutter speed of the image sensor 12 and the sample hold timing of the CDS circuit, 20 a microcomputer circuit for controlling the operation of the entire system (the AGC circuit of the AFE 13 is directly controlled), and 21 It is a communication control unit that transmits image data to a central monitoring room (not shown) via a network and outputs a control signal received from the central monitoring room side to the microcomputer circuit 20.

Although the basic configuration of the monitoring camera described above does not have a special feature, in this embodiment, the illumination condition at the time of shooting may be in a high contrast state such as backlight or over-order light. The DSP 16 is characterized in that an output frame is generated by analyzing a frame image and performing exposure control and specific image processing. Next, the operation procedure will be described with reference to the flowcharts of FIGS. 2a and 2b.

First, a captured image of the monitoring area is imaged on the image sensor 12 by the imaging optical system 11, and the image signal photoelectrically converted by the image sensor 12 undergoes signal processing by the AFE 13, and then is processed by the A / D converter 14. It is converted into a digital signal and input to the DSP 16. The DSP 16 saves the frame F (i) of the input image data in the memory 15 and executes a subject detection algorithm to check whether or not the subject is included in the frame F (i) (S1, S2). In this embodiment, a person is set as a subject, and the following algorithm can be adopted for the detection.
(1) Method for obtaining a person's position and area by detecting a person's face: Sampling image data of a predetermined size from an input frame, referring to pre-registered person's face feature data, The position and area of the person are determined by determining whether there is image data of the person's face.
(2) A method of calculating a difference between a background image registered in advance and a captured image, and analyzing the difference image to obtain the position and area of a person: a background image frame is acquired and registered in advance, and the captured image is acquired. The luminance information and color information of the difference image data between the frame and the background image frame are analyzed to determine the position and area of the person in the frame.
(3) Method of determining the person's area and position by motion detection: By calculating the luminance and color difference of the corresponding pixels in the previous frame and the current frame, and distinguishing the area where the difference is large from the background without motion Find the position and area of a person.

When a subject is detected by executing the subject detection algorithm, a luminance histogram of the subject region in frame F (i) of the memory 15 is created and saved in the memory 15 (S3, S4). Then, the distribution state (hereinafter referred to as “brightness distribution”) of the cumulative number of pixels in the luminance histogram is analyzed, and 20% or more of all the pixels related to the subject is a range between two predetermined luminance level thresholds. It is determined whether it is included in the intermediate luminance range (S5). This intermediate luminance range is a luminance range regarded as a proper exposure state. In this embodiment, for example, when the luminance level is expressed in 256 levels from 0 to 255, it is set to a range of 80 to 200. Here, when 20% or more of the pixels related to the subject are within the intermediate luminance range, an image with high contrast and high pixel resolution can be obtained at least for the subject itself. When 80% or more of the pixels are in the low luminance range (0 to 79) or the high luminance range (201 to 255), the image of the subject does not fall within the dynamic range of the image sensor 12, and the subject image is overexposed and underexposed. May occur.

In step S5, if 20% or more of all the pixels related to the subject are included in the intermediate luminance range, the DSP 16 creates exposure control data that matches the light amount of the entire frame F (i). Output to the lens / aperture drive unit 18, timing generator 19, and microcomputer circuit 20, and adjusts the aperture amount of the imaging optical system unit 11, the shutter speed of the image sensor 12, and the gain of AGC based on the exposure control data, respectively. Set (S5 → S6). The DSP 16 outputs the next frame F (i + 1) obtained in the control state to the D / A converter 17, and the image signal of the frame F (i + 1) is transmitted from the communication control unit 21 via the network. Is transmitted to the central monitoring room (S7). Further, when outputting the frame F (i + 1), the DSP 16 clears the save data in the memory 15 and then proceeds to the processing of the next frame F (i + 2) (S8, S9 → S1).

A specific image will be described as an example. FIG. 3A is a frame image obtained by photographing a subject (person) in the vicinity of an entrance and an indoor artificial flower or a houseplant in a direct light condition from the indoor side with the outdoors as a background. However, the luminance histograms relating to the entire image, the background region (including artificial flowers and foliage plants), and the subject region are as shown in FIGS. 4A, 4B, and 4C, respectively. In this case, in the luminance histogram of the subject area [FIG. 4C], all the luminance distributions are in the intermediate luminance range, and naturally, the processing is performed in the steps S6 and S7, but the entire frame F (i) is processed. Even when exposure control is performed in accordance with the amount of light, good contrast and visibility can be ensured on average. Note that the exposure control in step S6 is standard, and steps S6 and S7 are executed even if no subject is detected in step S2 (S3 → S6, S7).

On the other hand, in the captured image in the backlight state as shown in FIG. 3B, the pixels included in the intermediate luminance range among all the pixels related to the subject are smaller than 20%. In this case, the DSP 16 Immediately create a luminance histogram of the background region (region other than the subject) and divide the background region into a high luminance range, an intermediate luminance range, and a low luminance range in advance (S5 → S10, S11). Further, the DSP 16 performs a spatial frequency analysis for each of the areas belonging to the luminance ranges of the entire subject area and the background area detected in step S3, and saves the result in the memory 15 (S12). Then, the DSP 16 creates exposure control data for optimizing the brightness distribution based on the brightness histogram of the subject area created in step S4, and the lens / diaphragm drive unit 18 and the timing generator as described above. 19, the aperture amount of the imaging optical system unit 11, the shutter speed of the image sensor 12, and the gain of AGC are adjusted and set, and the next frame F (i + 1) obtained from the A / D converter 14 in the controlled state is stored in the memory. 15 is saved (S13, S14). The result of the spatial frequency analysis for the frame F (i) executed in step S12 will be described later, but it is compared with the result of each spatial frequency analysis for the frames F (i + 1) and F (i + 2). Thus, it is used when weighting processing is performed on the regions of the frames F (i + 1) and F (i + 2).

Next, the luminance distribution before the subject area is optimized in step S13 (that is, the luminance distribution of the luminance histogram of the subject area created in step S4) is unevenly distributed on either the low luminance range side or the high luminance range side. If it is unevenly distributed on the low brightness side, set the exposure state to optimize the brightness distribution of the high brightness range and intermediate brightness range related to the background area divided in step S9, and conversely If it is unevenly distributed on the high luminance side, an exposure state for optimizing the luminance distribution of the low luminance range and the intermediate luminance range related to the background region is set (S15 → S16orS17). In accordance with each case, the DSP 16 outputs exposure control data, and the lens / diaphragm drive unit 18 and the timing generator 19 adjust and set the aperture amount of the imaging optical system unit 11 and the shutter speed of the image sensor 12 to set the AGC. The gain is adjusted by the microcomputer circuit 20, but the next frame F (i + 2) obtained from the A / D converter 14 in the controlled state is saved in the memory 15 (S18).

Here, a specific consideration will be given to a case where the luminance distribution related to the subject becomes the determination result (the pixel included in the intermediate luminance range among all the pixels related to the subject is smaller than 20%). In this case, there is a possibility that the subject is present in a high-luminance area or a low-luminance area in the frame image. In particular, the possibility increases as the percentage decreases. For example, assuming a backlit state in the same shooting scene as in FIG. 3A, a frame image as shown in FIG. 3B is obtained, and the luminance histogram relating to the entire image of the frame, the background region, and the subject region is as follows. They are as shown in FIGS. 5A, 5B, and 5C, respectively. In other words, according to this example, the subject was photographed in a state in which the subject was completely blacked out in a high-intensity area where overexposure occurred outdoors, and the artificial flower was intermediate in the low-brightness area that was blacked out indoors. Although the image is taken as a portion of the luminance region, the foliage plant is blacked out in the low luminance region. Therefore, in the luminance histograms related to the entire frame image and the background region, the luminance distribution tends to be biased toward the low luminance range and the high luminance range as shown in FIGS. 5A and 5B, respectively. In each luminance histogram, as shown in FIG. 5C, most of the luminance distribution is concentrated in the low luminance range.

According to steps S13 and S14, the luminance distribution state of the subject shown in FIG. 5C (corresponding to the case where the luminance distribution before optimization of the subject region is unevenly distributed on the low luminance range side) is obtained by exposure control. As shown in FIG. 6A, the frame is raised to the intermediate luminance range side to obtain an image frame F (i + 1) as shown in FIG. 3C, and the image of the frame F (i + 1) is obtained. The overall luminance histogram has a luminance distribution as shown in FIG. Here, as is clear from a comparison between FIG. 3B and FIG. 3C, the low-luminance area that has been blacked out in FIG. As a result, the contrast of the image and the pixel resolution are increased so that the subject, the artificial flower and the foliage plant can be visually recognized. However, the state in which overexposure occurs in the high brightness area (outdoor) that is the background of the subject remains as it is, and the overexposure proceeds further when viewed from the brightness distribution of the brightness histogram in FIG. Will be. Although not shown, when the luminance distribution before the optimization of the subject area is unevenly distributed on the high luminance range side, reverse exposure control is performed to lower it to the intermediate luminance range side.

In steps S15 to S18, since the luminance distribution before optimization of the subject area is unevenly distributed to the low luminance range side, exposure control is performed to optimize the luminance distribution of the high luminance area and the intermediate luminance area of the background area. Taking the case of this as an example, an image frame F (i + 2) as shown in FIG. 3D is obtained, and a luminance histogram relating to the entire image of the frame F (i + 2) is shown in FIG. The luminance distribution is as shown in C). Specifically, the luminance distribution in the high luminance range and the intermediate luminance range in the luminance histogram in FIG. 5A corresponding to the captured image in FIG. 3B is lowered, and the background of the subject in FIG. The high luminance area which is changed to intermediate luminance, and the overexposure occurring in the high luminance area disappears in FIG. However, the blackening in the artificial flower portion and the low-brightness area, which are the intermediate luminance areas on the indoor side, remains as it is, and the blackening is more advanced as seen from the luminance distribution of the luminance histogram in FIG. Become.

When the saving of the frame F (i + 2) is completed (S18), the frames F (i + 1) and F (i + 2) are written in the memory 15, but the DSP 16 stores each frame F (i + 1). ), F (i + 2), the spatial frequency analysis is executed for each of the regions belonging to each luminance range in the entire subject region and the background region (S19, S20). Here, when setting the areas for the frames F (i + 1) and F (i + 2), the area information obtained in steps S3 and S11 for the frame F (i) is used. Although it is used as it is, since it can be considered that there is almost no change in the illumination conditions and the position of each region with a time difference of about two frames at most, there is substantially no problem.

By the way, both the above-described step S12 and the spatial frequency analysis in steps S19 and S20 determine the spectral intensity of the spatial frequency. First, in step S12, the spatial frequency analysis is executed for each region of the frame F (i). The result is as follows. Spatial periodicity can hardly be extracted from each image of the darkened subject area, the darkened low-luminance area of the background area, and the overexposed high-luminance area. Therefore, for example, the Fourier transform image for the subject region is in a state where the components are concentrated near the center as shown in FIG. 7A, and the spectrum intensity corresponding to each spatial frequency obtained as a result is shown in FIG. ) As a whole. On the other hand, in the background area where the brightness is intermediate and the contrast is high, the pattern appears relatively strong, so the Fourier transform image has components diffused as shown in FIG. Becomes stronger as shown in FIG. In the background area, when the area is on the high brightness area side, since it is overexposed as described above, the diffusion of components in the Fourier transform image is very small as shown in FIG. Becomes weak as shown in FIG.

On the other hand, since the luminance distribution of the subject area is optimized by setting the exposure state in step S13, the Fourier transform image of the subject area in the frame F (i + 1) is diffused as shown in FIG. The spectrum intensity by the spatial frequency analysis in step S19 should be much stronger as compared with FIG. 8A as shown in FIG. 8D. Further, since the brightness distribution of the background area is optimized so as to expand from the state unevenly distributed in the high brightness range or the low brightness range by the setting of the exposure state in steps S15 to S17, the frame F ( Also for the background region in i + 2), the diffusion of the Fourier transform image is large, and the spectral intensity by the spatial frequency analysis in step S20 should be strong as described above. Therefore, the DSP 16 performs weighting processing on the pixel data of each frame F (i + 1), F (i + 2) based on those results (S21, S22, S23). Specifically, in comparison with the frequency analysis for the frame F (i), the pixel data of the region in a more appropriate exposure state (that is, the region where the difference in spectral intensity is larger) is weighted more. The weighting process is executed according to the tendency to be performed. The weighting process does not necessarily need to cover the entire frequency range, but is limited to a frequency range [range given by f (0) to f (1) in FIG. 8] that is considered to be effective empirically. May be.

In the above example, with respect to the frame image [frame F (i + 1)] in FIG. 3C, a large weight is applied to the image data of the subject area and the indoor side image data of the background area. The high-intensity area that is the background tends to be weighted small because the amount of information as an image is small due to overexposure. On the other hand, with respect to the frame image [frame F (i + 2)] in FIG. 3D, a large weight is given to the indoor artificial flower portion and the background portion of the outdoor subject, which are the intermediate luminance regions, and other blackouts are made. In a low luminance region where the image is generated, the amount of information as an image is hardly obtained, and thus a small weight tends to be given.

When weighting processing is performed on the frames F (i + 1) and F (i + 2) in this way, the DSP 16 combines the image data of the frames F (i + 1) and F (i + 2). The generated frame Fcom is generated and output to the D / A converter 17 and transmitted to the centralized monitoring room side from the communication control unit 21 via the network in the same manner as in step S7, and the save data in the memory 15 is also transmitted. Clear (S24 ~ S26). When this frame synthesis is seen in the above example, for the actual image, the image obtained by applying the weighting process to the captured images of (C) and (D) of FIG. 3 is synthesized and shown in (E) of FIG. In addition, regarding the luminance histogram, the luminance distributions of (B) and (C) in FIG. 6 are changed by the weighting, and the luminance histogram after synthesis is shown in FIG. D). According to the distribution state of the cumulative number of pixels in FIG. 6D, there is almost no luminance distribution in the low luminance range and the high luminance range, and most of them are within the intermediate luminance range. In the frame image [frame Fcom], the subject, the artificial flower, and the foliage plant are displayed with good contrast and resolution without being blacked out or overexposed.

When the above-described procedure is repeatedly executed (S1 to S9 → S1, or S1 to S5 → S10 to S27 → S1), the state in which the subject is not detected from the frame image and the state in which the subject is in the proper exposure are continuously performed. If this is the case, one frame is output for every two frames for which exposure control is performed in accordance with the amount of light of the entire screen. This is not the case, and blackout or overexposure occurs due to backlight or excessive forward light. In the case where there is a continuous possibility that there is a possibility that the subject area and the background area are not blacked out or overexposed, one frame is output every three frames.

In this embodiment, the frame F (i) is used for subject detection and luminance distribution analysis, the frame F (i + 1) in which the luminance distribution in the subject region is optimized, and the frame in which the luminance distribution in the background region is optimized. F (i + 2) is obtained and these frames are synthesized. However, an appropriate exposure area of the frame F (i) may be used as image data at the time of frame synthesis, and in particular, the frame F (i ) Is used to obtain a frame F (i + 1) and synthesize it with the frame F (i) when it is confirmed that the exposure state is appropriate by analyzing the luminance distribution of the background area using it can. 2a and 2b assume that the subject is a single subject. However, when a plurality of subjects are detected, a brightness histogram is created for each subject, and the exposure state is determined from the brightness distribution. If the brightness distribution of each subject is almost the same, treat it as a single subject even if they are separated from each other. If the brightness distribution of each subject is different, optimize the exposure for each subject. The obtained frame is acquired, and the weighting process in steps S22 and S23 may be performed for each frame. Furthermore, in this embodiment, the weighting process is performed on the image data based on the analysis result on the luminance information. However, the weighting process may be performed using the analysis result on the color information together.

[Embodiment 2]
The case where the brightness is not sufficiently obtained in a part of the photographed image is not limited to the case where the blackout state is caused by the illumination condition such as the backlight as described above, and naturally, the brightness of the subject or the like is originally low. This also occurs in some cases (for example, when a person is dressed in black). In the exposure control and image processing in the surveillance camera according to the first embodiment, when the illumination condition at the time of shooting is in a high contrast state such as backlight or over-order light, a frame in which blackout or overexposure does not occur is generated. However, it does not distinguish the object with low brightness inherently, so if such an object exists in the captured frame, it may cause overexposure and conversely reduce the image quality. Become. In such a case, the surveillance camera of this embodiment automatically performs correction processing on the image data to prevent deterioration in image quality.

First, the configuration of the surveillance camera in this embodiment is the same as that shown in FIG. 1, and the DSP 16 analyzes the frame image under illumination conditions such as backlight and over-order light at the time of shooting, and is unique to exposure control. The generation of output frames by performing image processing is the same as in the first embodiment. However, the procedure of exposure control and image processing shown in FIG. 2b in the first embodiment (the procedure when the pixels in the intermediate luminance range are less than 20% in the luminance distribution of the subject) is executed according to the procedure shown in FIGS. 9a and 9b. Is different in that it is.

The characteristic processing contents in this embodiment will be described below with reference to FIGS. 9a and 9b. The procedure from step S10 to step S20 in FIG. 9a is basically a procedure for performing the processing described in FIG. 2b, and the numbers of the respective steps are also matched, but the procedures of steps S14a and S18a are inserted. Is different. That is, after the frame F (i + 1) is saved in the memory 16 in an exposure state that optimizes the luminance distribution of the subject area (S13, S14), a luminance histogram of the subject area of the frame F (i + 1) is created. And saving procedure (S14a), and after saving the frame F (i + 2) in the memory 16 in an exposure state that optimizes the luminance distribution of the background area (S15 to S18), A procedure (S18a) for creating and saving a luminance histogram of the background area of F (i + 2) is provided. Each luminance histogram created here is used in the later-described procedure (S30, S33).

Steps S30 to S40 in FIG. 9a are procedures specific to this embodiment. Further, according to the procedure from the previous step S1 to step S20 (however, from step S1 to step S9 is the same as in FIG. 2a), the memory 16 stores the [image data] and the [subject area of the subject area] in the frame F (i). [Brightness Histogram], [Background Area Luminance Histogram], [Spatial Frequency Analysis Result for the Area Subjected to All Luminance Ranges of Whole Subject Area and Background Area], [Image Data (Brightness of Subject Area) of Frame F (i + 1) Distribution is optimized)], [Brightness histogram of the subject area], [Spatial frequency analysis result for the whole subject area and the area belonging to each brightness range], and [Image data ( The luminance distribution of the background area is optimized), the luminance histogram of the background area, and the spatial frequency analysis result for the entire subject area and the areas belonging to the luminance ranges of the background area. Have been

In this embodiment, when the results of the spatial frequency analysis for the frames F (i + 1) and F (i + 2) are saved in the memory 16 in steps S19 and S20, they are created and saved in advance. The luminance histograms of the subject areas of the frames F (i) and F (i + 1) that have been compared are compared (S30). Then, in the comparison, it is determined whether or not the luminance distribution of the subject region has shifted from the low luminance range to the intermediate luminance range by a predetermined amount or more. If the luminance distribution cannot be confirmed, FIG. ) Is executed (S31 → S32). In this process (1), the result of the spatial frequency analysis for each subject area of the frames F (i) and F (i + 1) is compared, and whether or not the noise amount has increased to a predetermined value or more in the subject area. (S51, S52) and if there is such an increase, the amount of noise included in each subject area of the frames F (i), F (i + 1), F (i + 2) And the weighting process is executed with a tendency to apply a smaller weight to the image data of each subject area as the detected amount of noise increases (S52 → S53, S54). On the other hand, if there is no increase, the weighting process here is not performed (S52 → S33).

The processing from step S30 to step S32 will be described in relation to the result of the spatial frequency analysis as follows. First, when the subject itself is inherently low-luminance, such as when a person is dressed in black, the Fourier transform image of the subject region is compared for diffusion as shown in FIG. The spectral intensity also becomes weak as a whole as shown in FIG. This is due to the attribute of the subject itself, not according to the illumination conditions, and can hardly be changed even if the exposure state for optimizing the luminance distribution is set in step S13. On the other hand, since the setting of the exposure state in step S13 is performed based on the luminance histogram created in step S4, an overexposure condition is set. As shown in FIG. A lot of noise components are mixed. Therefore, if the noise component is greater than or equal to a predetermined value, the subject itself is considered to be inherently low-luminance, and by applying the weighting process, frames F (i), F ( This prevents noise from affecting the composite frame image of (i + 1) and F (i + 2).

The weighting process focusing on the noise is similarly executed for regions belonging to the luminance ranges of the background region. That is, in this case, the low luminance regions in the luminance histograms of the background regions of the frames F (i) and F (i + 2) created and saved in advance are compared (S33), and the luminance distribution is It is determined whether or not a predetermined amount or more has shifted from the low luminance range to the intermediate luminance range (S34), and if the shift of the luminance distribution cannot be confirmed, the process (2) shown in FIG. 9b (B) is executed. (S34 → S35). In this process (2), the results of spatial frequency analysis for the areas belonging to the luminance ranges of the background areas of the frames F (i) and F (i + 2) (hereinafter referred to as “target areas”) are compared. Then, it is determined whether or not there is an increase in the noise amount over a predetermined value in each target area (S61, S62), and if there is such an increase, the frames F (i), F The amount of noise included in the corresponding target area in (i + 1) and F (i + 2) is detected, and the larger the detected amount of noise for the image data in each corresponding target area, the more Weighting processing is executed with a tendency to perform small weighting (S62 → S63, S64). Therefore, as in the case of the process (1), it is possible to prevent noise from affecting the composite frame image obtained in the subsequent procedure (S38). Note that the weighting process is not performed when there is no increase as in the case of the process (1) (S52 → S36).

Next, processing is performed in the procedure from step S30 to step S35 in the areas belonging to the luminance ranges of the entire subject area and background area of the frames F (i), F (i + 1), and F (i + 2). The results of the spatial frequency analysis for the areas that did not exist are compared (S36), and for each unprocessed area, the weighting process is executed with a tendency to give a larger weight to the area in a more appropriate exposure state (S37). In other words, the same weighting process as in the first embodiment is performed at this stage, except for the area where the special process is performed in the procedure from step S30 to step S35.

Thereafter, the image data of each frame F (i), F (i + 1), F (i + 2) that has undergone the respective weighting processes is synthesized, and the synthesized frame Fcom is output to the D / A converter 17. The data is transmitted from the communication control unit 21 to the central monitoring room via the network, and the save data in the memory 15 is cleared (S38 to S40). Then, steps S1 to S9 in FIG. 2a and steps S10 to S20 and steps S30 to S40 in FIG. 9a are partially or entirely repeated depending on conditions (S41 → S1 to S9, S10 to S20). , S30-S41).

As a result, under the same frame output conditions as in the case of the first embodiment, it is possible to output a frame in which blackout and overexposure do not occur even in a backlight or over-order illumination state. According to this, when an object is inherently low in brightness as an attribute, the image area of the object is processed separately from the blackout, thereby eliminating the influence of noise associated with exposure control, and the object. An image frame that displays the brightness of itself as it is can be obtained. When the subject is a person, the low brightness area, the intermediate brightness area, and the high brightness area may be mixed in a single subject, such as black clothes but white face. In this case, it is only necessary to divide the area in accordance with the luminance of each area in the subject area and perform frame synthesis by weighting each area.

The present invention can be applied to a photographing apparatus such as a monitoring camera having an exposure control function.

It is a block diagram of the surveillance camera which concerns on embodiment of this invention. 6 is a flowchart showing an exposure control / output frame generation operation of the surveillance camera according to the first embodiment. 6 is a flowchart showing an exposure control / output frame generation operation of the surveillance camera according to the first embodiment. It is a figure which shows the example of the captured image and output image by a surveillance camera, (A) is the captured image in a follow light state, (B) is the captured image in a back light state, (C) is the brightness | luminance of a to-be-photographed object (person) area | region. The captured image in the exposure state with the optimized distribution, (D) is the captured image in the exposure state in which the luminance distribution of the background region is optimized, and (E) is the weighting process for the captured images in (C) and (D). This is an output image synthesized by applying. This is a luminance histogram when the surveillance camera is photographed in a normal light state, (A) is a luminance histogram of the entire captured image (frame), (B) is a luminance histogram of the background region, and (C) is a luminance histogram of the subject. . FIG. 6 is a luminance histogram when the surveillance camera is photographed in a backlight state, where (A) is a luminance histogram of the entire captured image (frame), (B) is a luminance histogram of the background region, and (C) is a luminance histogram of the subject. (A) is a luminance histogram of a subject area in a state where exposure of the subject area is optimized, (B) is a luminance histogram of a captured image in an exposure state where the luminance distribution of the subject area is optimized, and (C) is a background area. (D) is a luminance histogram of a composite image in an exposure state in which the luminance distribution is optimized. It is a Fourier transform image of the spatial frequency analysis result with respect to the image data which the surveillance camera image | photographed, (A) is a Fourier transform image which concerns on the to-be-photographed object area | region in a backlight condition, (B) is an intermediate brightness | luminance range within the background area | region in a backlight condition. And (C) is a Fourier transform image of a region belonging to the high luminance range in the background region in the backlight state, and (D) is a Fourier transform image of the subject region in the appropriate exposure state. , (E) are Fourier transform images of the subject area in the proper exposure state (when the subject itself is originally a low-luminance object). The spectral intensity of the spatial frequency analysis result for the image data captured by the surveillance camera, (A) is the spectral intensity related to the subject area in the backlight state, (B) belongs to the intermediate luminance range in the background area in the backlight state, And (C) is the spectral intensity of the region belonging to the high luminance range in the background region in the backlight state, (D) is the spectral intensity of the subject region in the proper exposure state, and (E) is This is the spectral intensity of the subject area in the proper exposure state (when the subject itself is originally a low-luminance object). 10 is a flowchart illustrating an exposure control / output frame generation operation of the monitoring camera according to the second embodiment. It is a flowchart which shows the procedure in each process of step S32 of FIG. 9a, and step S35. 6 is a graph showing a state in which the subject itself is inherently low-luminance and noise is mixed into the spectral intensity of the subject region due to overexposure by exposure control.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Imaging optical system part, 12 ... Image pick-up element, 13 ... AFE (Analogue Front End), 14 ... A / D converter, 15 ... Memory, 16 ... DSP (digital signal processing part), 17 ... D / A conversion , 18... Lens / aperture drive unit, 19... Timing generator, 20... Microcomputer circuit, 21.

Claims (4)

In a photographing device having an automatic exposure control function,
First frame storage means for storing first frame data of a captured image;
Subject detection means for detecting a target subject from the first frame data stored in the first frame storage means;
First histogram creation means for creating a brightness histogram of the subject area when the subject detection means detects a subject;
It is determined whether or not the distribution of the cumulative number of pixels in the luminance histogram of the subject area created by the first histogram creating means is included in a predetermined range or more within an intermediate luminance range set in advance as a luminance range regarded as a proper exposure state. Determination means to perform,
When the determination result of the determination unit is negative, a luminance histogram of a region other than the subject region (hereinafter referred to as “background region”) in the first frame data stored in the first frame storage unit is displayed. A second histogram creation means for creating;
When the determination result of the determination means is negative, the spatial frequency analysis for the subject area in the first frame data stored by the first frame storage means, the intermediate luminance range of the background area and the high First spatial frequency analysis means for performing spatial frequency analysis for each region belonging to the luminance range and the low luminance range;
When the determination result of the determination unit is negative, the exposure control is performed so that the distribution of the cumulative number of pixels in the luminance histogram of the subject area generated by the first histogram generation unit is substantially within the intermediate luminance range. First exposure control means for performing;
Second frame storage means for storing second frame data imaged in the exposure state set by the first exposure control means;
Second exposure control is performed so that the distribution of the cumulative number of pixels in the luminance histogram of the background region created by the second histogram creating means spreads from the state unevenly distributed in the high luminance range or the low luminance range to the intermediate luminance range side. Exposure control means;
Third frame storage means for storing third frame data captured in the exposure state set by the second exposure control means;
Spatial frequency analysis for the subject region in the second frame data stored by the second frame storage means, and spatial frequency analysis for each region belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region Second spatial frequency analysis means for performing
Spatial frequency analysis for the subject region in the third frame data stored by the third frame storage means, and spatial frequency analysis for each region belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region And a third spatial frequency analyzing means for executing
Each analysis result by the second spatial frequency analysis means and the third spatial frequency analysis means with respect to the analysis result by the first spatial frequency analysis means is respectively compared, and the second and third frame storage means respectively A weighting unit that applies a greater weighting process to the second and third frame data stored in each of the regions in a more appropriate exposure state,
Frame synthesizing means for synthesizing the second frame data of the second frame storage means and the third frame data of the third frame storage means after being weighted by the weighting means; And when the subject detection means detects the subject and the judgment result of the judgment means is negative, the frame composition means outputs the frame, and otherwise, the frame An image-taking apparatus that outputs a frame by performing automatic exposure control for setting an exposure state based on a total light amount. In a photographing device having an automatic exposure control function,
First frame storage means for storing first frame data of a captured image;
Subject detection means for detecting a target subject from the first frame data stored in the frame storage means;
First histogram creation means for creating a brightness histogram of the subject area when the subject detection means detects a subject;
It is determined whether or not the distribution of the cumulative number of pixels in the luminance histogram of the subject area created by the first histogram creating means is included in a predetermined range or more within an intermediate luminance range set in advance as a luminance range regarded as a proper exposure state. Determination means to perform,
A second histogram creating means for creating a luminance histogram of a background region in the first frame data stored by the first frame storage means when the judgment result of the judging means is negative;
When the determination result of the determination means is negative, the spatial frequency analysis for the subject area in the first frame data stored by the first frame storage means, the intermediate luminance range of the background area and the high First spatial frequency analysis means for performing spatial frequency analysis for each region belonging to the luminance range and the low luminance range;
When the determination result of the determination unit is negative, the exposure control is performed so that the distribution of the cumulative number of pixels in the luminance histogram of the subject area generated by the first histogram generation unit is substantially within the intermediate luminance range. First exposure control means for performing;
Second frame storage means for storing second frame data imaged in the exposure state set by the first exposure control means;
Third histogram creating means for creating a luminance histogram of the subject area in the second frame data stored by the second frame storing means;
Second exposure control is performed so that the distribution of the cumulative number of pixels in the luminance histogram of the background region created by the second histogram creating means spreads from the state unevenly distributed in the high luminance range or the low luminance range to the intermediate luminance range side. Exposure control means;
Third frame storage means for storing third frame data captured in the exposure state set by the second exposure control means;
Fourth histogram creating means for creating a luminance histogram of the background region in the third frame data stored by the third frame storing means;
Spatial frequency analysis for the subject region in the second frame data stored by the second frame storage means, and spatial frequency analysis for each region belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region Second spatial frequency analysis means for performing
Spatial frequency analysis for the subject region in the third frame data stored by the third frame storage means, and spatial frequency analysis for each region belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region And a third spatial frequency analyzing means for executing
The luminance histogram of the subject area in the first frame data created by the first histogram creating means is compared with the brightness histogram of the subject area in the second frame data created by the third histogram creating means. A first comparison / determination unit that determines whether or not the distribution of the cumulative number of pixels of the luminance histogram has shifted from the low luminance range to the intermediate luminance range by a predetermined amount or more;
When the determination result of the first comparison / determination means is negative, the analysis result of the subject area in the first frame data by the first spatial frequency analysis means and the second spatial frequency analysis A second comparison / determination means for comparing the result of analysis of the subject area in the second frame data by the means to determine whether or not the noise in the subject area has increased to a predetermined value or more;
If the determination result by the second comparison / determination means is affirmative, each subject area in the first to third frame data respectively stored in the first to third frame storage means is stored. A first weighting unit that detects a contained noise amount and applies a smaller weighting process to the image data of each subject area as the noise amount increases;
The brightness histogram of the background area in the first frame data created by the second histogram creating means is compared with the brightness histogram of the background area in the third frame data created by the fourth histogram creating means. A third comparison / determination means for determining whether or not the distribution of the cumulative number of pixels in the luminance histogram has shifted from the low luminance range to the intermediate luminance range by a predetermined amount or more;
When the determination result of the third comparison / determination means is negative, the intermediate luminance range, the high luminance range, and the low luminance of the background area in the first frame data by the first spatial frequency analyzing means The analysis result for each region belonging to the range (hereinafter referred to as “each target region”) is compared with the analysis result for each target region of the background region in the third frame data by the third spatial frequency analyzing means. And a fourth comparison / determination means for determining whether or not there is a noise that has increased above a predetermined value in each target region;
When the fourth comparison / determination means determines that there is a target region in which noise has increased to a predetermined value or more, the first to third frame storage means respectively store the first to third frames. Second weighting means for detecting a noise amount included in the corresponding target area of the frame data, and applying a smaller weighting process to the image data of the corresponding target area as the noise amount increases;
Among the analysis results by the first to third spatial frequency analysis means, compare the analysis results for each region excluded from the weighting processing target by the first weighting means and the second weighting means, A third weighting process is performed for each area of the first to third frame data respectively stored in the first to third frame storage units, in which a larger weighting process is performed in a more appropriate exposure state. Weighting means;
Frame synthesizing means for synthesizing the first to third frame data respectively stored in the first to third frame storage means subjected to weighting processing by the third weighting means. When the subject detection means detects a subject and the determination result of the determination means is negative, the frame composition means outputs a frame synthesized; otherwise, the entire frame is output. An image capturing apparatus that performs automatic exposure control for setting an exposure state based on a light amount and outputs a frame. A photographing method by a photographing device having an automatic exposure control function,
A first step of storing first frame data of a captured image in a first frame storage unit;
A second step of detecting a target subject from the first frame data stored in the first frame storage means;
A third step of creating a luminance histogram of the subject area when a subject is detected in the second step;
It is determined whether or not the distribution of the cumulative number of pixels in the luminance histogram of the subject area created in the third step is included at a predetermined ratio or more in the intermediate luminance range set in advance as the luminance range regarded as the proper exposure state. A fourth step;
A fifth step of creating a luminance histogram of the background region in the first frame data stored in the first frame storage means when the determination result in the fourth step is negative; ,
When the determination result in the fourth step is negative, the spatial frequency analysis for the subject region in the first frame data stored in the first frame storage unit, and the background region A sixth step of respectively performing a spatial frequency analysis for each region belonging to the intermediate luminance range, the high luminance range, and the low luminance range;
When the determination result in the fourth step is negative, exposure is performed so that the distribution of the cumulative number of pixels in the luminance histogram of the subject area created in the third step is substantially within the intermediate luminance range. A seventh step of controlling;
An eighth step of storing the second frame data imaged in the exposure state set in the sixth step in the second frame storage means;
Ninth step of performing exposure control so that the distribution of the cumulative number of pixels in the luminance histogram of the background region created in the fifth step spreads from the state unevenly distributed in the high luminance range or the low luminance range to the intermediate luminance range side. When,
A tenth step of storing the third frame data captured in the exposure state set in the ninth step in a third frame storage means;
Spatial frequency analysis of the subject area in the second frame data stored in the second frame storage means in the eighth step, and belongs to the intermediate luminance range, high luminance range, and low luminance range of the background region An eleventh step of performing each spatial frequency analysis for each region;
Spatial frequency analysis of the subject area in the third frame data stored in the third frame storage means in the tenth step, and the intermediate luminance range, high luminance range, and low luminance range of the background region A twelfth step of performing a spatial frequency analysis for each region,
The analysis results in the eleventh step and the twelfth step are compared with the analysis results in the sixth step, and the second and third frame storage means respectively store the analysis results. A thirteenth step of applying a larger weighting process to the second and third frame data in a region in a more appropriate exposure state;
A fourteenth frame for combining the second frame data of the second frame storage means and the third frame data of the third frame storage means after the weighting process is performed in the thirteenth step; And when the subject is detected in the second step and the determination result in the fourth step is negative, the frame synthesized in the fourteenth step is output, In other cases, the photographing method is characterized in that automatic exposure control for setting the exposure state based on the light amount of the entire frame is performed to output the frame. A photographing method by a photographing device having an automatic exposure control function,
A first step of storing first frame data of a captured image in a first frame storage unit;
A second step of detecting a target subject from the first frame data stored in the first frame storage means;
A third step of creating a luminance histogram of the subject area when a subject is detected in the second step;
It is determined whether or not the distribution of the cumulative number of pixels in the luminance histogram of the subject area created in the third step is included at a predetermined ratio or more in the intermediate luminance range set in advance as the luminance range regarded as the proper exposure state. A fourth step;
A fifth step of creating a luminance histogram of the background region in the first frame data stored in the first frame storage means when the determination result in the fourth step is negative; ,
When the determination result in the fourth step is negative, the spatial frequency analysis for the subject region in the first frame data stored in the first frame storage unit, and the background region A sixth step of respectively performing a spatial frequency analysis for each region belonging to the intermediate luminance range, the high luminance range, and the low luminance range;
When the determination result in the fourth step is negative, exposure is performed so that the distribution of the cumulative number of pixels in the luminance histogram of the subject area created in the third step is substantially within the intermediate luminance range. A seventh step of controlling;
An eighth step of storing the second frame data imaged in the exposure state set in the sixth step in the second frame storage means;
A ninth step of creating a luminance histogram of the subject area in the second frame data stored in the second frame storage means;
A tenth step of performing exposure control so that the distribution of the cumulative number of pixels in the luminance histogram of the background region created in the fifth step is spread to the intermediate luminance range side from a state of being unevenly distributed in the high luminance range or the low luminance range. When,
An eleventh step of storing third frame data captured in the exposure state set in the tenth step in a third frame storage means;
A twelfth step of creating a luminance histogram of a background region in the third frame data stored in the third frame storage means;
Spatial frequency analysis for the subject area in the second frame data stored in the second frame storage means, and space for each area belonging to the intermediate luminance range, high luminance range, and low luminance range of the background area A thirteenth step of performing each frequency analysis;
Spatial frequency analysis for the subject area in the third frame data stored in the third frame storage means, and space for each area belonging to the intermediate luminance range, high luminance range, and low luminance range of the background region A fourteenth step of performing frequency analysis respectively;
The luminance histogram of the subject region in the first frame data created in the third step is compared with the luminance histogram of the subject region in the second frame data created in the ninth step, and the luminance A fifteenth step of determining whether or not the distribution of the cumulative number of pixels of the histogram has shifted from the low luminance range to the intermediate luminance range by a predetermined amount or more;
If the determination result in the fifteenth step is negative, the analysis result on the subject area in the first frame data in the sixth step and the second result in the thirteenth step A sixteenth step of comparing the analysis result of the subject area in the frame data to determine whether or not the noise in the subject area has increased to a predetermined value or more;
If the determination result in the sixteenth step is affirmative, each of the subject areas in the first to third frame data respectively stored in the first to third frame storage means is included. A seventeenth step of detecting a noise amount, and applying a smaller weighting process to the image data of each subject area as the noise amount increases;
The luminance histogram of the background region in the first frame data created in the fifth step is compared with the luminance histogram of the background region created in the twelfth step, and the distribution of the cumulative number of pixels in the luminance histogram An eighteenth step of determining whether or not a predetermined amount or more has shifted from the low luminance range to the intermediate luminance range;
When the determination result in the eighteenth step is negative, each of the intermediate luminance range, high luminance range, and low luminance range of the background area in the first frame data in the sixth step belongs to The analysis result for the region (hereinafter, “each target region”) is compared with the analysis result for each target region of the background region in the third frame data in the fourteenth step. A nineteenth step of determining whether or not there is a noise increasing to a predetermined value or more;
The first to third frame data respectively stored in the first to third frame storage means when it is determined in the nineteenth step that there is a target region where noise has increased to a predetermined value or more. A twentieth step of detecting a noise amount included in the corresponding target region, and applying a smaller weighting process to the image data of the corresponding target region as the noise amount increases;
Of the analysis results of each frequency analysis in the sixth step, the thirteenth step, and the fourteenth step, each of the results that is excluded from the weighting processing target in the seventeenth step and the twentieth step The analysis results of the areas are compared, and the areas that are in a more appropriate exposure state with respect to the areas of the first to third frame data respectively stored in the first to third frame storage means. A 21st step of applying a larger weighting process;
And a twenty-second step of synthesizing the first to third frame data respectively stored in the first to third frame storage means subjected to the weighting process in the twenty-first step. If the subject is detected in the second step and the determination result in the fourth step is negative, the frame synthesized in the twenty-second step is output, otherwise A photographing method characterized in that automatic exposure control for setting an exposure state based on the light amount of the entire frame is performed to output the frame.

Images