JP6098575B2 - Imaging unit, image processing apparatus, and vehicle - Google Patents

Description

  The present invention relates to an image processing device, an imaging method, a program, and a vehicle.

  Sensing applications such as detection of lanes, etc., using technology to detect raindrops attached to the windshield and optical filters that transmit only the wavelength of the illumination light to automate wiper control when driving A technique for simultaneously detecting raindrops adhering to the windshield using the above camera has been known.

  For example, Patent Document 1 includes an image processing device, a light source, and an optical filter for the purpose of combining a camera for sensing use and a camera for detecting raindrops. An image processing system is disclosed that has a configuration that changes the intensity of the prepared illumination in order to capture the raindrops and follow the changing external light.

However, in such a conventional technique, a camera that combines a camera for sensing applications and a camera for detecting raindrops can follow an ever-changing external light and can perform imaging for use in raindrop detection processing . Since it was necessary to match the exposure, it was necessary to change the intensity of the illumination prepared in advance. For this reason, very powerful illumination is required, and there is a problem that power consumption increases.

  Further, in the prior art, strong light must be projected outside the vehicle, and further safety is required.

The present invention has been made in view of the above, and can reduce power consumption and improve safety even when the imaging unit is used for sensing and object detection. imaging unit capable, the image processing apparatus, the primary purpose of providing a contact and vehicle.

In order to solve the above-described problems and achieve the object, an imaging unit according to the present invention cuts light having a light source capable of irradiating light on a transparent body and light having a wavelength shorter than at least the light wavelength of the light source. It includes a filter, and an imaging unit that captures an area including the transparent body, the imaging unit to image the imaging area to be imaged without going through the filter when the light is not irradiated a first exposure amount for use in the light, characterized in that the switching between the second exposure used to image the imaging area to be imaged by through the filter Rutoki irradiated.

  According to the present invention, it is possible to reduce power consumption and improve safety even when an imaging unit for sensing and object detection is used.

Drawing 1 is a mimetic diagram showing a schematic structure of an in-vehicle device control system of an embodiment. FIG. 2 is a schematic diagram illustrating a schematic configuration of the imaging unit according to the embodiment. FIG. 3 is an explanatory diagram illustrating a schematic configuration of an imaging apparatus provided in the imaging unit of the embodiment. FIG. 4 is an explanatory diagram showing infrared light image data which is captured image data for raindrop detection when the imaging lens is focused on the raindrop on the outer wall surface of the windshield of the host vehicle of the embodiment. is there. FIG. 5 is an explanatory diagram illustrating infrared light image data that is captured image data for raindrop detection when the focus is set to infinity according to the embodiment. FIG. 6 is a graph showing the filter characteristics of the cut filter applicable to the captured image data for raindrop detection. FIG. 7 is a graph showing filter characteristics of a band-pass filter that can be applied to captured image data for raindrop detection. FIG. 8A is a side view of the optical filter. FIG. 8-2 is a front view of the front-stage filter provided in the optical filter of the imaging apparatus according to the embodiment. FIG. 9 is an explanatory diagram illustrating an example of captured image data of the imaging apparatus according to the embodiment. FIG. 10 is an explanatory diagram illustrating details of the imaging apparatus according to the embodiment. FIG. 11 is a schematic enlarged view when the optical filter and the image sensor of the imaging apparatus according to the embodiment are viewed from a direction orthogonal to the light transmission direction. FIG. 12 is a diagram for explaining foreign object detection. FIG. 13 is a diagram showing experimental results in a state where raindrops are attached. FIG. 14 is a diagram showing experimental results in a state where no raindrops are attached. FIG. 15 is a flowchart showing a procedure of exposure control processing according to the first embodiment. FIG. 16 is a relationship diagram between the light emission intensity of the light source and the exposure amount. FIG. 17 is a flowchart illustrating a control processing procedure according to the second embodiment. FIG. 18 is a flowchart illustrating a control processing procedure according to the third embodiment. FIG. 19 is a diagram illustrating an example of a captured image of a raindrop detection filter-equipped region when there is a foreign object such as a raindrop. FIG. 20 is a flowchart showing the procedure of exposure control processing according to the fifth embodiment. FIG. 21 is a configuration diagram of an optical system of the imaging unit of the third modification. FIG. 22 is a diagram illustrating an example of an image captured by the optical system according to the third modification.

  Hereinafter, embodiments of an image processing device, an imaging method, a program, and a vehicle will be described in detail with reference to the accompanying drawings. Note that the image processing apparatus according to the present invention is not limited to the in-vehicle device control system, and can be applied to, for example, other systems equipped with an object detection apparatus that detects an object based on a captured image.

(Embodiment 1)
FIG. 1 is a schematic diagram illustrating a schematic configuration of an in-vehicle device control system according to the first embodiment. The in-vehicle device control system uses the captured image data of the front area (imaging area) in the traveling direction of the host vehicle captured by the imaging device mounted on the host vehicle 100 such as an automobile, and controls the light distribution of the headlamps and drives the wiper. Control and control of other in-vehicle devices.

  As shown in FIG. 1, the in-vehicle device control system of the present embodiment includes an imaging unit 101 having an imaging device (not shown in FIG. 1, see FIG. 2), an exposure control unit 109, and an image analysis unit 102. The vehicle travel control unit 108, the wiper control unit 106, and the headlamp control unit 103 are mainly provided. Here, the vehicle travel control unit 108, the wiper control unit 106, and the headlamp control unit 103 function as a control unit that controls the operation of the host vehicle 100.

  The imaging device provided in the in-vehicle device control system of the present embodiment is provided in the imaging unit 101, and images the traveling region forward area of the traveling vehicle 100 as an imaging region. The windshield 105 is installed in the vicinity of a room mirror (not shown). The captured image data captured by the imaging device of the imaging unit 101 is input to the image analysis unit 102.

  The exposure control unit 109 performs exposure control of the imaging device of the imaging unit 101. A first exposure amount for imaging a first region in which an after-mentioned raindrop detection filter does not exist in the imaging region, and an exposure amount different from the first exposure amount, and a raindrop detection filter exists in the imaging region And a second exposure amount for imaging the second region to be imaged. The imaging apparatus switches between imaging of the imaging area with the first exposure amount determined by the exposure control unit 101 and imaging of the imaging area with the determined second exposure amount.

  The image analysis unit 102 analyzes the captured image data transmitted from the imaging device, calculates the position, direction, and distance of another vehicle existing ahead of the host vehicle 100 in the captured image data, or attaches to the windshield 105. For example, a detection object such as a white line (division line) on the road surface existing in the imaging region, or a rainfall amount is detected. In the detection of other vehicles, a preceding vehicle traveling in the same traveling direction as the own vehicle 100 is detected by identifying the tail lamp of the other vehicle, and traveling in the opposite direction to the own vehicle 100 by identifying the headlamp of the other vehicle. An oncoming vehicle is detected.

  The calculation result of the image analysis unit 102 is sent to the headlamp control unit 103. For example, the headlamp control unit 103 generates a control signal for controlling the headlamp 104 that is an in-vehicle device of the host vehicle 100 from the distance data calculated by the image analysis unit 102. Specifically, for example, while avoiding that the strong light of the headlamp of the own vehicle 100 is incident on the driver of the preceding vehicle or the oncoming vehicle, the driver of the other vehicle is prevented from being dazzled. The switching of the high beam and the low beam of the headlamp 104 is controlled, and partial shading control of the headlamp 104 is performed so that the driver's visibility can be secured.

  The calculation result of the image analysis unit 102 is also sent to the wiper control unit 106. The wiper control unit 106 controls the wiper 107 to remove deposits such as raindrops and foreign matters attached to the windshield 105 of the host vehicle 100. The wiper control unit 106 receives a foreign object detection result detected by the image analysis unit 102 and generates a control signal for controlling the wiper 107. When the control signal generated by the wiper control unit 106 is sent to the wiper 107, the wiper 107 is operated to ensure the visibility of the driver of the host vehicle 100.

  The calculation result of the image analysis unit 102 is also sent to the vehicle travel control unit 108. Based on the white line detection result detected by the image analysis unit 102, the vehicle travel control unit 108 warns the driver of the host vehicle 100 when the host vehicle 100 is out of the lane area defined by the white line. Notification is performed, and driving support control such as controlling the steering wheel and brake of the host vehicle 100 is performed.

  FIG. 2 is a schematic diagram illustrating a schematic configuration of the imaging unit 101. FIG. 3 is an explanatory diagram illustrating a schematic configuration of the imaging device 200 provided in the imaging unit 101. The imaging unit 101 includes an imaging device 200, a light source 202, and an imaging case 201 that accommodates these. The imaging unit 101 is installed on the inner wall surface side of the windshield 105 of the host vehicle 100. As shown in FIG. 3, the imaging device 200 includes an imaging lens 204, an optical filter 205, and an image sensor 206. The light source 202 emits light toward the windshield 105 and is arranged so that the reflected light enters the imaging device 200 when the light is reflected by the outer wall surface of the windshield 105.

  In the present embodiment, the light source 202 is for detecting an adhering matter adhering to the outer wall surface of the windshield 105 (hereinafter, the case where the adhering matter is a raindrop will be described as an example). When raindrops 203 are attached to the outer wall surface of the windshield 105, the light emitted from the light source 202 is reflected at the interface between the outer wall surface of the windshield 105 and the outside air, and the reflected light enters the imaging device 200. . Therefore, the raindrop 203 adhering to the windshield 105 is detected from the captured image data of the imaging device 200.

  In the present embodiment, the imaging unit 101 covers the imaging device 200 and the light source 202 with the imaging glass 201 together with the windshield 105 as shown in FIG. By covering with the imaging case 201 in this way, it is possible to suppress a situation where the windshield 105 covered with the imaging unit 101 is fogged even in a situation where the inner wall surface of the windshield 105 is clouded. Therefore, a situation where the image analysis unit 102 misanalyzes due to fogging of the windshield 105 can be suppressed, and various control operations based on the analysis result of the image analysis unit 102 can be appropriately performed.

  However, when the fogging of the windshield 105 is detected from the captured image data of the imaging device 200 and, for example, the air conditioning equipment of the host vehicle 100 is controlled, the portion of the windshield 105 facing the imaging device 200 is different from the other portions. A passage through which air flows may be formed in a part of the imaging case 201 so as to achieve the same situation.

  Here, in this embodiment, the focal position of the imaging lens 204 is set between infinity or infinity and the windshield 105. Thereby, not only when detecting the raindrop 203 attached on the windshield 105, but also when detecting the preceding vehicle and the oncoming vehicle and detecting the white line, appropriate information is obtained from the captured image data of the imaging device 200. Can be acquired.

  For example, when the raindrop 203 attached on the windshield 105 is detected, the shape of the raindrop image on the captured image data is often circular, so whether the raindrop candidate image on the captured image data is circular. A shape recognition process is performed to determine whether the raindrop candidate image is a raindrop image. When performing such shape recognition processing, the imaging lens 204 is focused on the raindrop 203 on the outer wall surface of the windshield 105, as described above, between infinity or infinity and the windshield 105. The in-focus state is slightly out of focus, the raindrop shape recognition rate (circular shape) is high, and the raindrop detection performance is high.

  FIG. 4 is an explanatory diagram showing infrared light image data that is captured image data for raindrop detection when the imaging lens 204 is focused on the raindrop 203 on the outer wall surface of the windshield 105. FIG. 5 is an explanatory diagram showing infrared light image data, which is captured image data for raindrop detection, when the focus is set to infinity. When the imaging lens 204 is focused on the raindrop 203 on the outer wall surface of the windshield 105, the background image 203a reflected in the raindrop is captured as shown in FIG. Such a background image 203 a causes erroneous detection of the raindrop 203. Further, as shown in FIG. 4, only a portion 203b of the raindrop may have a brightness that increases in a bow shape or the like. . In order to cope with the shape of such variously changing raindrop images by the shape recognition processing, the processing load is large and the recognition accuracy is lowered.

  On the other hand, when the image is focused at infinity, some blurring occurs as shown in FIG. For this reason, the reflection of the background image 203a is not reflected in the captured image data, and erroneous detection of the raindrop 203 is reduced. In addition, the occurrence of some defocusing reduces the degree of change in the shape of the raindrop image depending on the direction of sunlight, the position of the streetlight, and the like, and the shape of the raindrop image is always substantially circular. Therefore, the load of the shape recognition process of the raindrop 203 is small, and the recognition accuracy is high.

  However, when focusing at infinity, when identifying the tail lamp of a preceding vehicle traveling far, there may be about one light receiving element that receives the light of the tail lamp on the image sensor 206. In this case, as will be described in detail later, the light from the tail lamp may not be received by the red light receiving element that receives the tail lamp color (red). In this case, the tail lamp cannot be recognized and the preceding vehicle cannot be detected. In order to avoid such a problem, it is preferable that the imaging lens 204 is focused before infinity. As a result, the tail lamp of the preceding vehicle traveling far is out of focus, so that the number of light receiving elements that receive the light of the tail lamp can be increased, the recognition accuracy of the tail lamp is increased, and the detection accuracy of the preceding vehicle is improved.

  For the light source 202 of the imaging unit 101, a light emitting diode (LED), a semiconductor laser (LD), or the like can be used. Further, for example, visible light or infrared light can be used as the emission wavelength of the light source 202. However, in order to avoid dazzling the driver or pedestrian of the oncoming vehicle with the light of the light source 202, the wavelength is longer than the visible light and the light receiving sensitivity of the image sensor 206 reaches, for example, 800 nm or more. It is preferable to select a wavelength in the infrared light region of 1000 nm or less. The light source 202 of the present embodiment emits light having a wavelength in the infrared light region.

  Here, when infrared imaging light from the light source 202 reflected by the windshield 105 is imaged by the imaging device 200, the image sensor 206 of the imaging device 200 uses, for example, sunlight as well as infrared wavelength light from the light source 202. A large amount of disturbance light including the infrared wavelength light is also received. Therefore, in order to distinguish the infrared wavelength light from the light source 202 from such a large amount of disturbance light, it is necessary to make the light emission amount of the light source 202 sufficiently larger than the disturbance light. It is often difficult to use an amount of light source 202.

Therefore, in the present embodiment, for example, a cut filter that cuts light having a wavelength shorter than the emission wavelength of the light source 202 as shown in FIG. 6, or a peak of transmittance as shown in FIG. The light from the light source 202 is received by the image sensor 206 through a bandpass filter that substantially matches the emission wavelength of 202. As a result, light other than the emission wavelength of the light source 202 can be removed and received, so that the amount of light from the light source 202 received by the image sensor 206 is relatively large with respect to disturbance light. As a result, even if the light source 202 does not have a large light emission amount, the light from the light source 202 can be distinguished from disturbance light .

  However, in the present embodiment, not only the raindrop 203 on the windshield 105 is detected from the captured image data, but also the preceding vehicle and the oncoming vehicle and the white line are detected. Therefore, if the wavelength band other than the infrared wavelength light emitted by the light source 202 is removed from the entire captured image, the image sensor 206 receives light in the wavelength band necessary for detection of the preceding vehicle and oncoming vehicle and detection of the white line. This cannot be done and hinders these detections. Therefore, in the present embodiment, the image area of the captured image data, the raindrop detection image area for detecting the raindrop 203 on the windshield 105, the vehicle for detecting the preceding vehicle and the oncoming vehicle and the white line are detected. A filter that is divided into detection image regions and removes a wavelength band other than the infrared wavelength light emitted by the light source 202 only in a portion corresponding to the raindrop detection image region (hereinafter, sometimes referred to as “raindrop detection filter”). Is disposed in the optical filter 205. In other words, the raindrop detection image area is an area of captured image data obtained by imaging a region where a raindrop detection filter exists (hereinafter, sometimes referred to as a “raindrop detection filter area”). In addition, the vehicle detection image area is an area of captured image data obtained by capturing an area in which no raindrop detection filter exists in the imaging area (hereinafter, sometimes referred to as “raindrop detection filter-free area”).

  Here, the raindrop detection non-filter area corresponds to the first area, and the raindrop detection filter area corresponds to the second area.

  FIG. 8A is a side view of the optical filter 205. As shown in FIG. 8A, in the optical filter 205 of the present embodiment, a spectral filter layer 224 that transmits infrared light and visible light is formed on the surface of the substrate 221 on the imaging lens 204 side 205a. . In addition, a polarizing filter layer 222, an SOG (Spin On Glass) layer 223, and an infrared light transmission filter 212 that is a raindrop detection filter are sequentially formed on the surface of the substrate 221 on the image sensor 60 side 205b.

  By forming the filter layers on both surfaces of the substrate 221 of the optical filter 205 in this way, it is possible to suppress warping of the optical filter 205. When a multilayer film is formed only on one side surface of the substrate 221, stress is applied and warpage occurs. However, when multilayer films are formed on both surfaces of the substrate 221 as shown in FIG. 8A, warping can be suppressed because the effect of stress is offset.

  Here, the substrate 221 can be made of a material that is transparent to the visible light region, such as glass, sapphire, or quartz. In this embodiment mode, glass, in particular, inexpensive and durable quartz (refractive index 1.46) or Tempax glass (refractive index 1.51) is used as the material of the substrate 221.

  The spectral filter layer 224 formed on the surface of the substrate 221 on the optical filter 205a side is a filter that transmits both a so-called visible light region having a wavelength range of 400 nm to 670 nm and an infrared light region having a wavelength range of 940 to 970 nm. is there. The visible light region is used to detect vehicle periphery information, and the infrared light region is used to detect raindrops. The spectral filter layer 224 does not transmit light in the wavelength range of 700 to 940 nm (the transmittance is preferably 5% or less). This is because, when this wavelength range is taken in, the obtained image data becomes entirely red, and it may be difficult to extract a portion showing the red color of the tail lamp. Therefore, if a filter having the characteristic of cutting infrared light is formed, light of other colors that becomes a disturbance can be removed, so that, for example, detection accuracy of a tail lamp can be improved.

  The polarizing filter layer 222 formed on the surface on the optical filter 205b side is a filter that cuts the S-polarized component and transmits only the P-polarized component. By this polarizing filter layer 222, disturbance factors and unnecessary reflected light (reflection light) can be cut.

  In the present embodiment, the polarizing filter layer 222 is formed of a wire grid polarizer. The wire grid is formed by arranging conductor wires made of a metal such as aluminum in a lattice pattern at a specific pitch, and the pitch is larger than that of incident light (for example, visible light wavelength 400 nm to 800 nm). If the pitch is very small (for example, less than one half), the light of the electric field vector component that oscillates parallel to the conductor line is almost reflected, and the light of the electric field vector component perpendicular to the conductor line is reflected. Because it is almost transparent, it can be used as a polarizer that produces a single polarization.

  In the wire grid polarizer, when the cross-sectional area of the metal wire is increased, the extinction ratio is increased. Further, the transmittance of the metal wire having a predetermined width or more with respect to the periodic width is decreased. Further, when the cross-sectional shape orthogonal to the longitudinal direction of the metal wire is a tapered shape, the wavelength dispersion of the transmittance and the polarization degree is small in a wide band, and a high extinction ratio characteristic is exhibited. In the cross-sectional structure (not shown) of the wire grid, light with a polarization direction in the groove direction is shielded, and light with a polarization direction in a direction perpendicular to the groove is transmitted.

  In this embodiment, since the wire grid structure is used as the polarizer of the polarizing filter layer 222, there are the following advantages. In other words, the wire grid structure may be a well-known semiconductor process, that is, an aluminum thin film is deposited and then patterned to form the sub-wavelength uneven structure of the wire grid by a technique such as metal etching. Therefore, since the direction of the polarizer can be adjusted corresponding to the pixel size of the image sensor (several micron level), the transmission polarization axis can be selected in units of pixels as in this embodiment. Moreover, since the wire grid structure is made of a metal such as aluminum as described above, it has excellent heat resistance and is suitable for in-vehicle use.

  In the SOG layer 223 of this embodiment, the substrate transparent to the light in the use band and the convex portions of the wire grid extending linearly on the substrate are arranged at a pitch smaller than the wavelength of the light in the use band. Between the aluminum convex portions, a filling portion filled with an inorganic material having a refractive index lower than or equivalent to that of the substrate 221 is formed. The filling portion is formed so as to cover the convex portion of the wire grid structure.

The material for forming the SOG layer 223 is preferably a low refractive index material whose refractive index is as close as possible to the refractive index of air = 1 so as not to deteriorate the polarization characteristics of the polarizer of the polarizing filter layer 222. For example, a porous ceramic material formed by dispersing fine pores in ceramics is preferable, and examples include porous silica (SiO ₂ ), porous magnesium fluoride (MgF), and porous alumina (Al ₂ O ₃ ). It is done. The degree of these low refractive indexes is determined by the number and size (porosity) of the holes in the ceramic. Among these, in particular, when the main component of the substrate 221 is made of silica crystal or glass, porous silica (n = 1.2-1.26) is preferable because the refractive index is smaller than that of the substrate 221. .

As a method for forming the SOG layer 223, an inorganic coating film (SOG: Spin On Glass) generation method is used. That is, it is formed in such a manner that a solvent in which silanol [Si (OH) ₄ ] is dissolved in alcohol is spin-coated on a substrate, and then a solvent component is volatilized by heat treatment to cause silanol itself to undergo a dehydration polymerization reaction.

  By using the SOG layer 223, since the polarizing filter layer 222 has a sub-wavelength sized wire grid structure, it is weaker in strength than the infrared light transmission filter 212 formed on the SOG layer 223. In particular, since the optical filter 205 is desirably disposed in close contact with the image sensor 206, there is a possibility that the optical filter 205 and the image sensor surface of the image sensor 206 are in contact with each other, but the polarizing filter layer is weak in strength. Since 222 is protected by the SOG layer 223, the optical filter 205 can be mounted without damaging the wire grid structure. Note that the spectral filter layer 224 can also be protected by the SOG layer 223.

  Further, by providing the SOG layer 223, entry of foreign matter into the wire grid portion can be suppressed. The height of the convex portion of the wire grid is generally configured to be half or less of the wavelength used. On the other hand, the spectral filter is several times as high as the wavelength used, and the transmittance characteristics at the cutoff wavelength can be sharpened as the thickness is increased. Further, as the thickness of the SOG layer 223 increases, it is difficult to ensure the flatness of the upper surface, and it is not appropriate to increase the thickness because the homogeneity of the filling region is impaired. In this embodiment, since the infrared light transmission filter 212 is formed after the polarizing filter layer 222 is covered with the SOG layer 223, the SOG layer 223 can be stably formed. The infrared light transmission filter 212 formed on the upper surface of the SOG layer 223 can also be optimally formed.

  FIG. 8B is a front view of the optical filter 205 on the image sensor 60 side. FIG. 9 is an explanatory diagram illustrating an example of captured image data. As shown in FIG. 8B, an area 211 arranged at a location corresponding to the center of the captured image (part corresponding to the height 2/4 of the imaging area), which is the vehicle detection image area 213, and raindrop detection Infrared light transmission disposed at locations corresponding to the upper portion of the captured image (portion corresponding to the imaging region height ¼) and the lower portion of the captured image (portion corresponding to the imaging region height ¼) as the image region 214 The filter 212 is divided into regions. As the infrared light transmission filter 212, the cut filter shown in FIG. 6 or the band pass filter shown in FIG. 7 is used.

  In many cases, the headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line image are mainly present in the upper part of the captured image, and the image of the nearest road surface in front of the host vehicle is usually present in the lower part of the captured image. Therefore, the information necessary for identifying the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line is concentrated on the upper portion of the captured image, and the information on the lower portion of the captured image is not so important in the identification. Therefore, when both oncoming vehicle, preceding vehicle, or white line detection and raindrop detection are performed simultaneously from a single captured image data, as shown in FIG. It is preferable to provide the region 214 and the remaining central portion of the captured image as the vehicle detection image region 213, and to provide the infrared light transmission filter 212 corresponding to the region 214.

  When the imaging direction of the imaging apparatus 200 is tilted downward, the hood of the host vehicle may enter the lower part of the imaging area. In this case, sunlight reflected by the bonnet of the host vehicle or the tail lamp of the preceding vehicle becomes disturbance light, which is included in the captured image data, which may cause misidentification of the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line. Become. Even in such a case, in this embodiment, the cut filter shown in FIG. 6 and the bandpass filter shown in FIG. Disturbance light such as the tail lamp of the vehicle is removed. Therefore, the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line identification accuracy are improved.

  In this embodiment, due to the characteristics of the imaging lens 204, the scene in the imaging area and the image on the image sensor 206 are upside down. Therefore, when the lower portion of the captured image is used as the raindrop detection image region 214, the upper portion of the upstream filter 210 of the optical filter 205 is configured by the cut filter shown in FIG. 6 or the bandpass filter shown in FIG. .

  Here, when the preceding vehicle is detected, the preceding vehicle is detected by identifying the tail lamp on the captured image. However, the tail lamp has a smaller amount of light compared to the headlamp of the oncoming vehicle, and disturbance such as street lights. Since there is a lot of light, it is difficult to detect the tail lamp with high accuracy only from mere luminance data. Therefore, it is necessary to identify the tail lamp based on the amount of received red light using spectral information for identifying the tail lamp. Therefore, in this embodiment, as will be described later, a red filter or a cyan filter (a filter that transmits only the wavelength band of the tail lamp color) that matches the color of the tail lamp is disposed in the subsequent filter 220 of the optical filter 205, and the red color. The amount of light received can be detected.

  However, since each light receiving element constituting the image sensor 206 of the present embodiment has sensitivity to light in the infrared wavelength band, it is obtained when the image sensor 206 receives light including the infrared wavelength band. The captured image becomes reddish as a whole. As a result, it may be difficult to identify the red image portion corresponding to the tail lamp. Therefore, in the present embodiment, the portion corresponding to the vehicle detection image region 213 in the upstream filter 210 of the optical filter 205 is the infrared light cut filter region 211. Thereby, since the infrared wavelength band is excluded from the captured image data portion used for identifying the tail lamp, the identification accuracy of the tail lamp is improved.

  FIG. 10 is an explanatory diagram showing details of the imaging apparatus 200 in the present embodiment. The imaging apparatus 200 mainly includes an imaging lens 204, an optical filter 205, a sensor substrate 207 including an image sensor 206 having a two-dimensionally arranged pixel array, and an analog electrical signal ( The signal processing unit 208 generates and outputs captured image data obtained by converting the received light amount received by each light receiving element on the image sensor 206 into a digital electric signal. Light from the imaging region including the subject (detection target) passes through the imaging lens 204, passes through the optical filter 205, and is converted into an electrical signal corresponding to the light intensity by the image sensor 206. In the signal processing unit 208, when an electrical signal (analog signal) output from the image sensor 206 is input, the brightness (luminance) of each pixel on the image sensor 206 is shown as captured image data from the electrical signal. The digital signal is output to the subsequent unit together with the horizontal / vertical synchronizing signal of the image.

  FIG. 11 is a schematic enlarged view when the optical filter 205 and the image sensor 206 are viewed from a direction orthogonal to the light transmission direction. The image sensor 206 is a light receiving element using a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, and uses a photodiode 206A for each pixel. The photodiodes 206A are two-dimensionally arranged for each pixel, and a microlens 206B is provided on the incident side of each photodiode 206A in order to increase the light collection efficiency of the photodiode 206A. The image sensor 206 is bonded to a PWB (Printed Wiring Board) by a method such as wire bonding to form a sensor substrate 207.

  The optical filter 205 and the image sensor 206 may be bonded with a UV adhesive, or UV bonding or thermocompression may be performed in the four-side region outside the effective pixel while the outside of the effective pixel range used for photographing is supported by a spacer or the like. By tightly joining the optical filter 205 and the image sensor 206, the boundary between the raindrop detection area and the vehicle detection area becomes clear, and the accuracy of determining the presence or absence of raindrops is increased.

  Further, in the image area shown in FIG. 9, when the raindrop detection area is provided only on either the upper part or the lower part of the screen, it is difficult to bond the optical filter 205 and the image sensor 206 in parallel, and the inclination is inclined. It will adhere. If they are bonded at an inclination, the optical path length changes between the upper part of the region and the lower part of the region, and this causes deterioration of recognition accuracy such as reading errors of white line coordinates when detecting vehicle peripheral information, for example, white line coordinates. For this reason, in the present embodiment, as shown in FIG. 9, the raindrop detection area is provided at the upper and lower parts of the captured image, and the infrared light transmission filter 212 is provided corresponding to this, so that the optical filter 205 and It becomes easy to bond the image sensors 206 in parallel, and as a result, the accuracy of raindrop detection can be improved.

  Next, foreign object detection for controlling the wiper 107 and the washer (not shown) according to the present embodiment is possible. Here, the foreign matter is raindrops, but in addition to this, it is also possible to include bird droppings and splashes on the road surface from the adjacent vehicle.

  FIG. 12 is a diagram for explaining foreign object detection. The light source 202 is arranged so that the regular reflection light on the outer surface of the windshield substantially coincides with the optical axis of the imaging lens. The light rays received by the imaging apparatus 200 will be described with reference to the light rays A to E in the drawing.

  Light A: The light A emitted from the light source 202 and passing through the windshield leaks to the outside as it is when raindrops are not attached to the outside as viewed from the imaging device 200 of the windshield 105. As the light source 202, a light source having a wavelength / light quantity in the eye-safe band is selected, and safety is further improved by outputting light upward as shown in FIG.

  Light B: A part of the light emitted from the light source 202 is reflected on the surface of the light when entering the windshield 105. It is generally known that the polarization component is an S polarization component. Such light becomes unnecessary light for the original raindrop detection and causes erroneous detection. However, in this embodiment, the S-polarized component is cut by the polarizing filter layer 222 of the optical filter 205. Therefore, unnecessary light can be removed.

  Light C: Of the light emitted from the light source 202, the component of the light C transmitted through the windshield 105 without being reflected by the inner surface of the windshield 105 has a larger P-polarized component than the S-polarized component. When the raindrops are attached to the outside of the windshield 105, the light incident on the windshield 105 is multiple-reflected inside the raindrops and is transmitted through the windshield again toward the image pickup apparatus 200 to take an image. It reaches the optical filter 205 of the device. Then, since the groove direction of the wire grid structure is formed so as to transmit the spectral filter layer 224 and the P polarizing component in the subsequent polarizing filter layer 222, it also passes through this, and at this time, in the raindrop detection region An infrared light transmission filter 212 is formed in accordance with the wavelength of the light source 202, but the light ray C passes therethrough, reaches the image sensor 206, and recognizes that raindrops are attached to the surface of the windshield 105. be able to.

  Light D: Of the light that reaches the imaging device 200 from the outside of the windshield 105 instead of the light source 202, most of the light D that reaches the raindrop detection region is cut by the infrared light transmission filter. In this way, the raindrop detection area is configured to cut out disturbance light outside the windshield 105.

  Ray E: The ray E that passes through the region other than the raindrop detection region, that is, the region where the infrared light transmission filter 212 does not exist, transmits only the visible region and infrared light, and only P-polarized components are cut off unnecessary light. It reaches the image sensor 206 in a state and is detected as a signal for various applications.

  The incident angle to the windshield 105 is set so that the light source 202 may image the reflected light on any one of the boundary surfaces between raindrops and air. As shown in FIG. 12, the layout in which the reflected light from the raindrop is the strongest is the case where it is installed at a position substantially opposite to the optical axis of the imaging device 200 with respect to the normal of the surface of the windshield 105, and the imaging device. In this case, the optical axis is almost the same as the optical axis 200, and the reflected light is the smallest when the normal of the windshield 105 and the optical axis of the light source substantially coincide.

  The light source 202 can be arranged so as to irradiate only the region of the infrared light transmission filter 212. Thereby, the noise component from the vehicle detection area can be avoided. A plurality of light sources 202 may be provided. In this case, the polarizer pattern of each region of the polarizing filter layer 222 includes the optical axis of light emitted from the light source having the largest incident light quantity to the polarizer pattern out of the plurality of light sources 202 toward the windshield 105, and imaging. It is set so as to transmit only the polarization component parallel to the surface formed by the two optical axes of the lens and the optical axis.

  As a light emission method of the light source 202, pulse light emission may be performed at a specific timing in addition to continuous light emission (also referred to as CW light emission). By synchronizing the timing of light emission and the timing of photographing, the influence of disturbance light can be further reduced. When a plurality of light sources 202 are installed, the plurality of light sources 202 may emit light simultaneously or sequentially. When light is emitted sequentially, the influence of disturbance light can be further reduced by synchronizing the light emission timing with the photographing timing.

  Here, the experimental results taken by the inventor are shown in FIGS. The lower and upper portions of the image area are raindrop detection areas. FIG. 13 is a diagram showing experimental results in a state where raindrops are attached. FIG. 14 is a diagram showing experimental results in a state where no raindrops are attached.

  The portion described as “detection area” in the image is an area for detecting raindrops. When raindrops adhere to this area, LED light from the light source 202 enters, and as shown in FIG. If not, LED light is not detected. When the raindrop is attached, the raindrop is recognized so as to be displayed as “Rain detected”. Further, when it is not attached, it indicates a state of not being recognized by “Rain not detected”. This recognition process can be easily performed by adjusting the threshold value of the amount of received LED light. Note that the threshold value need not be uniquely determined, and the optimum value may be calculated based on the exposure adjustment information of the vehicle detection area of the imaging apparatus 200.

  Next, the exposure control process according to this embodiment will be described. FIG. 15 is a flowchart showing a procedure of exposure control processing according to the first embodiment. First, the exposure control unit 109 determines a first exposure amount for imaging a raindrop detection filter non-region (corresponding to the vehicle detection image region 213), which is a region 211 where the infrared light transmission filter 212 is not present ( Step S11).

  The first exposure amount is an exposure amount suitable for imaging the raindrop detection filter-free region (vehicle detection image region 213).

  Here, the exposure control unit 109 determines the first exposure amount based on the amount of light in the raindrop detection non-filter area. In other words, the first exposure amount is determined by setting the measurement range in the automatic exposure control (AE: Auto Explosure) to the raindrop detection filter-free region. Specifically, the exposure amount is determined by the exposure time.

  That is, when shooting a distant image, automatic exposure adjustment is performed while detecting a portion of the raindrop detection filter-free region. In the region 211 where the infrared light transmission filter 212 does not exist, the peripheral light amount change is large. The illuminance around the vehicle imaged by the imaging apparatus 200 varies from tens of thousands of lux during the day to 1 lux or less at night. For this reason, it is necessary to adjust the exposure time according to the shooting scene, and for this, known automatic exposure control is performed. In the present embodiment, since the subject is around the road surface, it is preferable to perform exposure control based on the image of the road surface area.

  Next, the exposure control unit 109 determines a second exposure amount for imaging a raindrop detection filter area (corresponding to the raindrop detection image area 214), which is an area where the infrared light transmission filter 212 exists ( Step S12).

  This second exposure amount is an exposure amount suitable for imaging the raindrop detection filter-equipped region (raindrop detection image region 214). Since the raindrop detection filter-equipped region is set so as to capture only the reflected light from the foreign matter of the light source 202, the change in the amount of light due to the surrounding environment is small. For this reason, it is also possible to shoot with a fixed exposure time that does not depend on the situation of external light.

  FIG. 16 is a relationship diagram between the light emission intensity of the light source 202 and the exposure amount. As shown in FIG. 16, as the emission intensity increases, the exposure amount decreases (exposure time decreases). In the present embodiment, the exposure control unit 109 changes and determines the second exposure amount from the drive current corresponding to the light emission intensity of the light source according to the relationship of FIG.

  As described above, imaging with the first exposure amount suitable for the raindrop detection filter-free region results in imaging of the raindrop detection filter-free region. Imaging with the second exposure amount suitable for the raindrop detection filter-equipped region results in imaging of the raindrop detection filter-equipped region. Therefore, hereinafter, imaging with the first exposure amount suitable for the raindrop detection filter-free region is referred to as “raindrop detection filter-free region imaging”, and the second exposure amount suitable for the raindrop detection filter-equipped region. This imaging may be referred to as “imaging the raindrop detection filter area”.

  Next, the exposure control unit 109 determines the number n of continuous imaging with the first exposure amount (step S13). Further, the exposure control unit 109 determines the number m of continuous imaging with the second exposure amount (step S14). Here, the number m of continuous imaging at the second exposure amount is set to a number smaller than the number n of continuous imaging at the first exposure amount. For example, n = 30, m = 1, and the like.

  Then, the imaging apparatus 200 continuously performs imaging with the first exposure amount n times (step S15). Then, the image analysis unit 102 analyzes the captured image (step S16) and performs various controls by the headlamp control unit 103, the vehicle travel unit 108, and the like (step S17).

  Next, the imaging apparatus 200 continuously performs imaging with the second exposure amount m times (step S18). Then, the image analysis unit 102 analyzes the captured image (step S19) and performs various controls by the wiper control unit 106 and the like (step S20).

  Then, the processing from step S15 to S20 is repeatedly executed until there is a predetermined termination instruction (such as a termination instruction by the user) (step S21: No).

  For example, when n = 30 and m = 1, the imaging apparatus 200 continuously performs imaging suitable for the raindrop detection filter-free region at the first exposure amount 30 times, and then at the second exposure amount. Switching is performed such that imaging suitable for the raindrop detection filter area is performed once, and then imaging suitable for the raindrop detection filter non-area at the first exposure amount is performed 30 times continuously.

  The image of the raindrop detection filter-free region is used by the image analysis unit 102 for recognition processing such as white line detection and vehicle detection. Since these recognition processes also use information between input image frames, it is necessary to input frames taken at predetermined time intervals or rules to the image analysis unit 102.

  In addition, raindrop detection performed to determine whether or not to drive the wiper does not change the situation in a short time compared to, for example, detection of lane departure or front inter-vehicle distance, and also gives priority to safety. The degree is low.

  For this reason, the number m of continuous imaging of the raindrop detection filter area is determined so that the captured image of the raindrop detection filter area is inserted at regular intervals into the captured image of the raindrop detection filter area. It is determined to be smaller than the number n of continuous imaging of the area.

  As described above, in the present embodiment, the imaging device 200 captures both the captured image for vehicle detection and the captured image for raindrop detection. However, continuous imaging of the captured image of the raindrop detection filter-free region is performed. Since the image analysis is performed by switching the continuous imaging of the raindrop detection filter area, the power consumption can be reduced and the safety can be further improved.

(Embodiment 2)
The in-vehicle device control system of the second embodiment shortens the insertion cycle of the captured image of the raindrop detection filter-equipped region when raindrops are detected, that is, for raindrop detection with respect to the number of continuous imaging of the raindrop detection filter-free region The ratio of the number of times of continuous imaging in the filter area is increased. Specifically, when raindrops are detected, the imaging apparatus 200 decreases the number of consecutive imaging n of the raindrop detection filter-free area and does not change the number of consecutive imaging m of the raindrop detection filter-equipped area, or The number of continuous imaging n of the raindrop detection filter-free region is not changed, and the number of continuous imaging m of the raindrop detection filter-equipped region is increased.

  The schematic configuration of the in-vehicle device control system of the present embodiment and the schematic configuration of the imaging unit 101 and the imaging device 200 are all the same as those of the first embodiment.

  Since it is rare that a large amount of rain falls suddenly, the frequency of frames for photographing the raindrop detection filter area may be reduced when no raindrop is detected. However, once a raindrop is detected, it is desirable to look closely at the change in rainfall. In this case, the imaging apparatus 200 increases the frequency of frames for photographing the raindrop detection filter area. doing.

  For example, imaging with no raindrop detection filter x120 times → imaging with raindrop detection filter x1 times → imaging with raindrop detection filter x120 times → imaging x1 with raindrop detection filter region → raindrop detection filter Let us consider a case where switching such as non-region imaging x 120 times →. In this case, when a raindrop is detected once, the frequency is changed so as to look closely as follows.

  Raindrop detection filter no region imaging x30 times → Raindrop detection filter no region imaging x1 times → Raindrop detection filter no region imaging x30 times → Raindrop detection filter no region imaging x1 → Raindrop detection filter no region no region Imaging x 30 times →.

  In the present embodiment, when a raindrop is detected, the wiper control unit 106 drives the wiper at a speed corresponding to the amount of raindrop. Then, the imaging apparatus 200 performs imaging of the raindrop detection filter area, that is, imaging with the second exposure amount, in synchronization with the wiper driving cycle of the wiper control unit 106.

  That is, since the raindrops are wiped after the wiper is driven, the raindrops cannot be detected. For this reason, the raindrop detection filter area is imaged at the timing when the wiper finishes wiping. Thereby, it is not necessary to increase the number of unnecessary raindrop detection imaged images, and the number of frames of the vehicle or white line detection imaged image can be increased. In addition, when the state where raindrops are not detected continues for a certain period of time, it is preferable to return the imaging frequency of the raindrop detection filter area to the normal frequency.

  FIG. 17 is a flowchart illustrating a control processing procedure according to the second embodiment. The image analysis unit 102 detects raindrops from the captured image of the raindrop detection filter-equipped region (step S31). If raindrops are detected (step S32: Yes), the imaging apparatus 200 increases the ratio of the number m of continuous imaging of the raindrop detection filter-equipped area to the number of continuous imaging n of the raindrop detection filter-free area (step S32). S33).

  Next, the wiper control unit 106 determines the driving speed of the wiper according to the raindrop amount (step S34), and the wiper is driven based on this.

  Next, the imaging apparatus 200 determines the timing of imaging at the second exposure amount suitable for the raindrop detection filter-equipped region as the timing synchronized with the wiper driving described above (step S35). Thus, the raindrop detection filter area is imaged at the timing when the wiper finishes wiping.

  As described above, in this embodiment, when raindrops are detected, the ratio of the number m of continuous imaging of the raindrop detection filter area to the number n of continuous imaging counts of the raindrop detection filter-free area is increased. Since the insertion period of the captured image in the filter area is shortened, it is possible to grasp the change in rainfall with high accuracy.

  In the present embodiment, since the raindrop detection filter area is imaged at the timing when the wiper finishes wiping, there is no need to increase the number of unnecessary raindrop detection images. The number of frames of the captured image can be increased.

(Embodiment 3)
The light quantity around the vehicle can be measured by acquiring the first exposure amount (exposure time) in the raindrop detection filter-free region in which automatic exposure is performed. When the first exposure amount is small (exposure time is short), it can be estimated that the weather is clear because the surroundings are bright. Since it is unlikely that it will rain in fine weather, it is desirable to reduce the imaging frequency of the raindrop detection filter area.

  For this reason, in this embodiment, when it is determined that the sky is clear from the first exposure amount of the raindrop detection filter-free area, the raindrop detection filter-equipped area continues for the number n of continuous imaging of the raindrop detection filter-free area. By reducing the ratio of the number of times of imaging m, the insertion period of the captured image of the raindrop detection filter-equipped region is lengthened.

  The schematic configuration of the in-vehicle device control system of the present embodiment and the schematic configuration of the imaging unit 101 and the imaging device 200 are all the same as those of the first embodiment.

  Specifically, in the present embodiment, the imaging apparatus 200 increases the number of continuous imaging n of the raindrop detection filter-free area and does not change the number of continuous imaging m of the raindrop detection filter-equipped area, or The number of continuous imaging n of the raindrop detection filter-free region is not changed, and the number of continuous imaging m of the raindrop detection filter-equipped region is decreased.

  In the present embodiment, a light source control unit (not shown) in the imaging unit 101 changes the light emission intensity of the light source 202 according to the first exposure amount. Basically, in this embodiment, it is not necessary to change the light amount, but in the case of excessively strong external light, the light emission intensity is changed as follows.

  When the amount of light changes (for example, the amount of light is about 0.5 lux to 100,000 lux), the raindrop detection filter area is affected by outside light and the background of the portion where no foreign matter is attached It may change, and in the worst case, the entire region is saturated and foreign matter cannot be detected.

  In this case, the light source control unit of the imaging unit 101 needs to project from the light source 202 with light stronger than the external light that has passed through the infrared light transmission filter 212 in the raindrop detection filter-equipped region. It is preferable to calculate how much light enters the raindrop detection filter area from the exposure amount of the area without the filter area and change the power of the light source according to the amount.

  In this case, the first exposure amount in the raindrop detection filter-free region may be changed in accordance with the light emission intensity of the changed light source.

  On the contrary, since the outside light hardly enters at night, the light emission intensity of the light source 202 may be small. Also in this case, the light source control unit of the imaging unit 101 calculates how much light enters the raindrop detection filter area from the first exposure amount in the raindrop detection filter non-area portion. However, by changing the light emission intensity of the light source 202 according to the amount, the light emission intensity can be reduced, and low power consumption can be realized.

  FIG. 18 is a flowchart illustrating a control processing procedure according to the third embodiment. First, the imaging unit 101 measures the amount of light from the first exposure amount as described above (step S41). Then, the imaging unit 101 determines whether or not the weather is sunny (step S42). If the weather is sunny (step S42: Yes), the raindrop detection filter area with respect to the continuous imaging count n of the raindrop detection filter non-area The ratio of the continuous imaging frequency m is reduced (step S43).

  Next, the light source control unit of the imaging unit 101 changes the light emission intensity of the light source 202 as described above according to the amount of light (step S44).

  As described above, in the present embodiment, in the case of fine weather, by reducing the ratio of the number m of continuous imaging in the raindrop detection filter area to the number n of continuous imaging in the raindrop detection filter-free area, Since the insertion period of the captured image in the filter-equipped area is lengthened, it is possible to reduce the imaging frequency of the raindrop detection filter-equipped area in fine weather and to reduce power consumption.

  In this embodiment, since the light emission intensity of the light source 202 is changed according to the first exposure amount, the power consumption can be further reduced.

(Embodiment 4)
In the present embodiment, in addition to the above-described embodiments, when raindrops are detected by the image analysis unit 102, the measurement of the rainfall is performed using the value relating to the dispersion of the pixel values of the captured image of the raindrop detection filter area as an index. I do.

  The schematic configuration of the in-vehicle device control system of the present embodiment and the schematic configuration of the imaging unit 101 and the imaging device 200 are all the same as those of the first embodiment.

  In the image of the raindrop detection filter-equipped region, when there is a foreign object such as a raindrop, reflected light is generated to obtain a captured image shown in FIG. In the example of FIG. 19, when the standard deviation value of the brightness of the SWS camera photographed image is calculated, it becomes 20, 27, 39 from the left image, respectively, and it can be seen that it correlates with the rainfall.

  For this reason, in the present embodiment, a rain amount calculation unit (not shown) of the image analysis unit 102 calculates a value related to dispersion such as a standard deviation value of luminance from the captured image of the raindrop detection filter-equipped region. Rainfall is measured based on dispersion values. Note that the value relating to the variance includes the variance itself.

  Note that the amount of reflected light may be simply used as an indicator of rainfall, but in that case, it is greatly affected by fluctuations in the light emission intensity of the light source 202. For this reason, it is better to obtain an index of rainfall from a value related to dispersion of reflected light. For example, in the case where there are raindrops, since the density appears in the image of the raindrop detection filter area, the variance value becomes large. In addition, the greater the amount of rain, the greater the number of raindrops and the larger the raindrops.

  As described above, in the present embodiment, the rainfall calculation unit of the image analysis unit 102 measures the rain using the value relating to the dispersion of the pixel values of the captured image of the raindrop detection filter-equipped region as an index. High accuracy can be achieved.

  Further, in the present embodiment, the rain amount calculation unit of the image analysis unit 102 calculates a value related to dispersion by excluding a luminance value equal to or less than a predetermined threshold among pixel values of a captured image of the raindrop detection filter-equipped region. Alternatively, the image analysis unit 102 may be configured.

  Even in the region with the raindrop detection filter, a slight amount of outside light that cannot be blocked by the infrared light transmission filter 212 appears in the image of the region with the raindrop detection filter as a background. In order to eliminate the influence, before calculating the variance, it is preferable to exclude a luminance value equal to or lower than a predetermined threshold value of the image of the raindrop detection filter area from the calculation of the value related to the variance.

  For example, before calculating the standard deviation value, if the standard deviation value is calculated after subtracting 128 values for each pixel (or 0 if the reduction result is 0 or less), The standard deviation is 6, 16, 29 from the image, and it can be seen that the correlation with the rainfall is greater.

  Therefore, this has an advantage that it is possible to further improve the accuracy of rainfall measurement.

(Embodiment 5)
In the present embodiment, the raindrop detection filter area is imaged while controlling irradiation and non-irradiation of irradiation light from the light source 202, and the amount of rain is detected from the obtained captured image.

  The schematic configuration of the in-vehicle device control system of the present embodiment and the schematic configuration of the imaging unit 101 and the imaging device 200 are all the same as those of the first embodiment.

  Even in the raindrop detection filter-equipped region (second region), since external light includes light having the same wavelength as that of the light source 202, even when the light source 202 is turned off (that is, when light is not irradiated), External light leaking from the filter enters the image sensor 206. By mixing external light into the raindrop detection filter area, the brightness value obtained by the intensity of external light changes regardless of the amount of rain, and the accuracy of raindrop detection deteriorates.

  For this reason, in the present embodiment, in the imaging of the raindrop detection filter area by the imaging device 200, the light source 202 is turned on (that is, the light is irradiated) and the light source 202 is turned off (that is, the light is turned off). Two captured images in a non-irradiated state are taken in succession, and the image analysis unit 102 performs processing to cancel the leaked outside light.

  In the present embodiment, a light source control unit (not shown) is provided in the imaging unit 101, and the light source control unit controls light irradiation from the light source 202 to turn off or turn on the light source 202. To do.

  In imaging of the raindrop detection filter-equipped region (second region), the imaging apparatus 200 performs first imaging in a light-off state where light is not emitted from the light source 202, and second imaging in a lighting state where light is emitted from the light source 202. Switch between and.

  Considering that the vehicle travels and the leaked outside light changes sequentially, the shorter the time between the two captured images, the better. Therefore, the captured image with the light source 202 turned on and the light source 202 turned off. Is preferably a continuous frame. Therefore, more specifically, the imaging apparatus 200 performs the first imaging and the second imaging alternately in succession.

  In addition, when the first captured image and the second captured image are captured with the same exposure amount, the image analysis unit 102 can improve the accuracy of the external light cancellation process most. The first imaging and the second imaging are performed with the second exposure amount that is the same exposure amount.

  Further, the image analysis unit 102 is provided with a raindrop detection unit (not shown) that detects the amount of raindrops attached to the windshield 105. The raindrop detection unit performs external light cancellation processing by subtracting the pixel value of the first captured image obtained by the first imaging from the pixel value of the second captured image obtained by the second imaging in the lighting state. .

  More specifically, the raindrop detection unit calculates the pixel value of the pixel of the first captured image by the first imaging in the unlit state from the sum of the pixel values of the second captured image by the second imaging in the lit state. The total light is subtracted to cancel the external light, and the subtracted total value is used as a rainfall index to detect the rainfall.

  In the present embodiment, as described in Embodiment 1 with reference to FIGS. 4 and 5, the imaging lens 204 is not focused on the outer wall surface of the windshield 105, so that it adheres to the outer wall surface of the windshield 105. The obtained captured image is a circle having a substantially constant size regardless of the area. However, since there is a correlation between the area of the raindrop and the luminance value of the obtained captured image, the size of one raindrop can be estimated from the luminance value.

  When a large number of raindrops are attached to the windshield 105, a large number of circles having luminance values corresponding to the size of the captured image are obtained. Therefore, in the present embodiment, as a value having a correlation with the area of the surface on which raindrops are attached, the sum of the luminance values of the range of the captured image of the raindrop detection filter-equipped region, that is, the captured image of the raindrop detection filter-equipped region The sum of the pixel values is used as an indicator of rainfall.

  In this case, the raindrop detection unit can also perform external light cancellation processing by subtracting the luminance value of the first captured image for each pixel from the luminance value of the second captured image. In this case, in order to subtract for each pixel, it is necessary to store all the pixel values of the captured image in the off state, and the memory capacity of the memory is required by the number of pixels in the raindrop detection filter area, The device is also expensive.

  In contrast, the sum of the pixel values (luminance values) of the first captured image in the unlit state is stored in a memory or the like, and the raindrop detection unit detects the pixel values of the second captured image when the light source 202 is lit. Since the sum of the pixel values (luminance values) of the first captured image when the light source 202 is turned off is subtracted from the sum of the (luminance values), the sum of the pixel values of the respective captured images can be calculated. Compared with the case where subtraction is performed every time, a frame buffer is not necessary, and the storage capacity of the memory can be reduced.

  Next, the exposure control process according to this embodiment will be described. FIG. 20 is a flowchart showing the procedure of exposure control processing according to the fifth embodiment. First, the processing from the determination of the first exposure amount to the determination of the number m of continuous imaging with the second exposure amount (steps S51 to S54) is the same as the processing from steps S11 to S14 in the exposure control processing of the first embodiment. The same is done.

  Then, as in the first embodiment, the imaging apparatus 200 continuously performs imaging with the first exposure amount in the raindrop detection filter-free region n times (step S55). Then, the image analysis unit 102 analyzes the captured image (step S56), and performs various controls by the headlamp control unit 103, the vehicle travel control unit 108, and the like (step S57).

  Next, the imaging device 200 executes the imaging of the raindrop detection filter-equipped region once with the second exposure amount with the light source 202 turned off (step S58). Furthermore, the imaging apparatus 200 performs imaging of the raindrop detection filter-equipped region once with the second exposure amount with the light source 202 turned on (step S59).

  Next, the raindrop detection unit of the image analysis unit 102 performs raindrop detection after performing external light cancellation processing by the method described above (step S60). Such processes from step S58 to S60 are repeatedly executed m / 2 times. Then, various controls by the wiper control unit 106 and the like are performed (step S61).

  Then, the processes from step S55 to S61 are repeatedly executed until there is a predetermined end instruction (such as an end instruction by the user) (step S62: No).

  For example, in the case of n = 30 and m = 2, the imaging apparatus 200 continuously performs imaging suitable for the raindrop detection filter-free region with the first exposure amount 30 times. Thereafter, the imaging apparatus 200 performs imaging of the raindrop detection filter-equipped region once with the second exposure amount with the light source 202 turned off. Subsequently, the imaging apparatus 200 continuously turns on the light source 202. The raindrop detection filter area is imaged once with the second exposure amount. After that, the imaging apparatus 200 continuously performs imaging 30 times suitable for the raindrop detection filter-free region with the first exposure amount. Then, the imaging apparatus 200 captures the raindrop detection filter-equipped region once with the second exposure amount with the light source 202 turned off. Subsequently, the imaging apparatus 200 continuously turns on the light source 202. Switching is performed such that the raindrop detection filter area is imaged once with the second exposure amount. Note that the imaging frequency of the raindrop detection filter-free region may be increased.

  As described above, in the present embodiment, imaging of the raindrop detection filter-equipped region is performed once with the second exposure amount with the light source 202 turned off. Subsequently, the imaging apparatus 200 turns on the light source 202. In this state, the raindrop detection filter area is imaged once with the second exposure amount, and the raindrop detection is executed by canceling the outside light. Therefore, the raindrop detection is reduced by reducing the influence of the outside light. It can be realized with accuracy.

  Note that either imaging with the light source 202 turned off or imaging with the light source 202 turned on may be performed first.

  As mentioned above, although Embodiment 1-5 was demonstrated, various changes or improvement can be added to the said embodiment.

(Modification 1)
The rain detection unit subtracts the sum of the pixel values (luminance values) of the first captured image when the light source 202 is turned off from the sum of the pixel values (luminance values) of the second captured image when the light source 202 is turned on. A non-linear characteristic may be imparted to the value, that is, the subtracted sum value, and the non-linear characteristic may be imparted to detect the rainfall based on the value.

  In the case of detecting the rain amount by the subtracted total value, a large number of small raindrops having the same area are detected as a larger rain amount than a single large raindrop. For this reason, in this modification, a more accurate rainfall index can be obtained by adding a non-linear component to the subtracted sum value.

  As an example of providing the non-linear characteristic, there is a method of calculating an index of rainfall by the following formula, but the present invention is not limited to this.

  rain_mount = exp (sum (img (x, y) × coeff1-1))

Here, rain_mount: Rainfall index img (x, y): Luminance value of the raindrop detection filter area sum (): Sum operator of pixel values of the raindrop detection filter area coeff1: Coefficient

  As the coefficient coeff1, an appropriate value can be obtained from experiments.

(Modification 2)
When the raindrop detection unit performs the external light cancellation process by subtracting the pixel value of the first captured image in the unlit state from the pixel value of the second captured image in the lit state for each pixel, the amount of rain The detection unit adds a nonlinear characteristic to a value obtained by subtracting the pixel value of the first captured image from the pixel value of the second captured image for each pixel, obtains a total value of the values imparted with the nonlinear characteristic, and calculates the total value It may be configured to detect rainfall based on the above. Thereby, it is possible to detect the rainfall more accurately.

  As an example of providing the non-linear characteristic, there is a method of calculating an index of rainfall by the following formula, but the present invention is not limited to this.

rain_mount = sum (exp (img (x, y) * coeff2-1))
Here, rain_amount: rain amount index img (x, y): luminance value of the raindrop detection filter area sum (): summation operator of pixel values of the raindrop detection filter area coeff2: coefficient

  As the coefficient coeff2, an appropriate value can be obtained from experiments.

(Modification 3)
In the first to fifth embodiments, the optical system of the imaging unit 101 uses the one shown in FIG. In the third modification, a total reflection optical system is used for the imaging unit 101. FIG. 21 is a configuration diagram of an optical system of the imaging unit 101 of the third modification.

  In the imaging unit 101 of the present embodiment, the light emitted from the light source 202 is incident on the reflection deflection member 2101 provided on the inner surface of the windshield 105 as shown by the optical path in FIG. The light is totally reflected by the outer wall surface of the windshield 105 and enters the imaging lens 2041, and is received by the image sensor 20 of the sensor substrate 2042.

  When a total reflection optical system as shown in FIG. 21 is used, the acquired captured image is inverted as compared with the optical system of the imaging unit 101 of the first to fifth embodiments. FIG. 22 is a diagram illustrating an example of an image captured by the optical system according to the third modification. As can be seen from the example of FIG. 22, the captured image of the example shown in FIG. 19 in the fourth embodiment is inverted. That is, the place where raindrops are attached becomes dark, and the place where no raindrops are attached becomes bright.

  In the optical system of the imaging unit 101 of the first to fifth embodiments, the value of the processing result increases as the amount of rain increases. On the contrary, in the optical system of the present modification using total reflection, the processing result increases as the amount of rain increases. The value becomes smaller. However, the captured image analysis processing is performed in the same manner as the processing in the first to fifth embodiments.

  Also, in order to make the rainfall, which is the processing result of the rainfall detection unit of this modification, easier to handle by wiper control etc., the rainfall is compared with a threshold value, for example, based on the following: The rain detection unit may be configured so as to be sent to the wiper control unit 106.

Rainfall Output to wiper control unit Meaning 0-0.1 0 No rain 0.1-0.3 1 Light rain 0.3-0.5 2 Rain 0.5-0.7 3 Heavy rain 0.7-1.0 4 very heavy rain

  As a result, the wiper control unit 106 receives five levels from 0 to 4, and provides easy-to-handle information with meaning. In addition, the amount of data is reduced and efficient compared to the case where the rainfall value is transmitted as it is.

  Each process executed by the in-vehicle device control system of the above-described embodiment can be realized by hardware, and can also be realized by a program. In this case, each processing program executed by the in-vehicle device control system of the above embodiment is provided by being incorporated in advance in a ROM or the like.

  Each processing program executed in the in-vehicle device control system of the above embodiment is a file in an installable format or an executable format, and is a CD-ROM, flexible disk (FD), CD-R, DVD (Digital Versatile Disk). For example, the program may be recorded on a computer-readable recording medium.

  Furthermore, each processing program executed in the in-vehicle device control system of the above embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. . Further, each processing program executed by the in-vehicle device control system of the above embodiment may be provided or distributed via a network such as the Internet.

  Each processing program executed in the in-vehicle device control system according to the above embodiment has a module configuration including the above-described units or units, and as actual hardware, a CPU (processor) stores each processing program from the ROM. By reading and executing the above, each of the above parts is loaded onto the main storage device, and each part or each unit is generated on the main storage device.

DESCRIPTION OF SYMBOLS 100 Own vehicle 101 Imaging unit 102 Image analysis unit 103 Headlamp control unit 104 Headlamp 105 Windshield 106 Wiper control unit 107 Wiper 108 Vehicle travel control unit 200 Imaging device 201 Imaging case 202 Light source 203 Raindrop 204 Imaging lens 205 Optical filter 206 Image Sensor 206A Photodiode (light receiving element)
211 Raindrop detection no filter area 212 Infrared light transmission filter area (raindrop detection filter area)
213 Image area for vehicle detection 214 Image area for raindrop detection 221 Substrate 222 Polarizing filter layer 223 SOG

JP-A-2005-195 566

Claims (9)

A light source capable of emitting light to a transparent body;
An imaging unit that has a filter that cuts light having a wavelength shorter than the wavelength of light of the light source, and that captures an imaging region including the transparent body;
With
The imaging unit may through the filter the a first exposure used to image the imaging area to be imaged without going through the filter, the Rutoki the light is irradiated when the light is not irradiated imaging unit, characterized in that for switching the second exposure used to image the imaging area to be imaged Te. Imaging at the first exposure amount per unit time is greater than imaging at the second exposure amount per unit time;
The imaging unit according to claim 1. The imaging unit includes an image sensor partially including the filter,
Performing imaging with the first exposure amount in a portion of the image sensor where the filter does not exist, and performing imaging with the second exposure amount in a portion of the image sensor where the filter exists;
The imaging unit according to claim 1. The imaging unit switches between imaging with a first exposure amount that changes in accordance with the amount of light in the imaging region and imaging with a second exposure amount that is constant regardless of the amount of light in the imaging region;
The imaging unit according to any one of claims 1 to 3. The imaging unit switches between imaging with a first exposure amount that changes according to the amount of light in the imaging region and imaging with a second exposure amount that changes according to the light emission intensity of the light source;
The imaging unit according to any one of claims 1 to 3. The imaging unit switches between imaging with a first exposure amount that changes according to the amount of light in the imaging region and imaging with a second exposure amount that changes according to the drive current of the light source;
The imaging unit according to claim 5. An image processing apparatus comprising the imaging unit according to any one of claims 1 to 6,
Based on an image acquired by imaging the first exposure amount, an object located far from the transparent body is recognized, and on the transparent body based on an image acquired by imaging the second exposure amount. Further having an image analysis unit for recognizing an object;
An image processing apparatus.   The said image analysis part outputs the result of the recognition process of the object on the said transparent body as a step-wise level value determined according to the recognition degree of the object on the said transparent body, It is characterized by the above-mentioned. 8. The image processing apparatus according to 7. A vehicle comprising the image processing device according to claim 7 or 8,
The image analysis unit performs raindrop recognition processing on a windshield based on an image acquired by imaging the second exposure amount,
A wiper control unit that controls a wiper for wiping the windshield based on a result of the raindrop recognition process;
A vehicle characterized by