DE102011015056A1 - Optical device for use in windscreen of motor car to recognize pedestrian at edge of lane during rain, has reflective element reflecting radiation from optical paths having field angles larger than maximum field of view angle - Google Patents

Abstract

The device has a lens system (5) for projecting electromagnetic radiation from an object-side region to an optical axis (7) on an image sensor (6) at a maximum field of view angle (a1). A reflective element (10) i.e. mirror, reflects the electromagnetic radiation from optical paths having field angles with respect to the image sensor, where the field angles are larger than the maximum field of view angle. The reflective element is arranged perpendicular to the optical axis, and formed by surface structures i.e. repetitive wedge structures, or diffraction grating. An independent claim is also included for a method for evaluating an image data.

Description

The invention relates to an optical device for imaging electromagnetic radiation on a sensitive semiconductor surface and to a method for evaluating image data that is recorded with the optical device.

In the prior art optical image pickup devices are known, which consist of the following components:

Lens element:

At least one element for imaging the electromagnetic radiation by refraction (lens or prism), reflection (mirror or prism) or diffraction (diffractive element). This element is referred to below as a lens system.

Wherein this single element or system may contain a geometrically delimiting the beam path element, which is referred to below as a diaphragm. However, the aperture can also be positioned independently of the lens system.

The lens system can also be designed so that an intermediate image takes place. In the plane of the intermediate image diaphragm structures, wavelength-dependent filter coatings, transmission filters or polarization coatings can be provided.

These functional elements can be designed to be electronically controllable or static.

Image sensor:

At least one sensor element which is sensitive to the desired spectral range of the electromagnetic radiation. This element is referred to below as an image sensor. The image sensor can be designed as a matrix structure of sensitive semiconductor elements (pixels). Wherein in a further level of the image sensor (between sensitive semiconductor elements and lens system), a wavelength-dependent filter array can be applied, which allows a color distinction in the image.

Likewise, in another embodiment, a polarization-dependent filter array could be applied in this plane.

A combination of polarization- and wavelength-dependent filter array can be advantageously used to identify certain properties of the emanating from an object point or an object surface electromagnetic radiation.

Additionally, the image sensor may include an array of microlenses that refractively, refractively or reflectively geometrically focus electromagnetic radiation to obtain a higher irradiance on the sensitive semiconductor surface.

In the following, a pixel of an image sensor is understood to be the sensitive semiconductor surface and, if appropriate, the associated signal preprocessing, wavelength-dependent or polarization-dependent filter or filter pixel, microlens, which are assigned to the sensitive semiconductor surface of this pixel.

The image sensor consists of an array of single pixels, possibly protected by a cover glass.

The lens element and the image sensor require a maximum field of view angle from which electromagnetic radiation can be imaged on the image sensor. In a camera structure according to the prior art, the parameters image sensor width and focal length of the lens element result according to the following equation a maximum field angle α, which can be detected by the image sensor:

Strictly speaking, α could be termed the maximum horizontal field of view angle, and a maximum vertical field of view angle could be determined by replacing the image sensor width with the image sensor height. Hereinafter, however, "maximum field angle" is meant as indicated in Equation 1.

Electromagnetic radiation that radiates from object-side or entry-side field angles that are greater than the maximum field angle α of a camera system can not be detected in a spatially resolved manner and can only be estimated as scattered or stray light by the entire image sensor.

For many applications, it is desirable or necessary to extend the maximum field of view that can be detected with an image sensor with a given optical system (lens, aperture, etc.).

In order to increase the maximum field of view angle for a given lens element, image sensors having a larger area or higher resolution have hitherto been used. Both lead to significantly higher costs.

It is an object of the present invention to provide an optical device which allows the cost of imaging a larger field of view and a method for evaluating image data taken with the device.

The invention provides a solution with which even field angles are imaged in a spatially resolved manner on the image sensor, which are larger than the maximum field of view angle.

By means of an optical device according to the invention, a similar imaging is achieved as by a camera system with a larger image sensor.

The spatially resolved imaging on the image sensor takes place by reflecting beam paths with field angles greater than the maximum field of view angle onto at least one area of the image sensor, which causes the mirrored image to overlap with the image which, without the reflection, ie. H. directly, would result.

To solve the task so the reflection of beam paths with field angles greater than the maximum field of view angle is proposed on the image sensor. For this purpose, according to the invention a reflective element is provided, for. B. a mirror. Preferably, the reflective element has a flat surface, whereby the reflection takes place directed. Alternatively, the reflective surface or the reflective element is not planar, but rather represents a free-form surface. The free-form surface can be specially designed to distort, enlarge or correct image aberrations.

An optical device according to the invention can, for. B. in a motor vehicle behind the windshield enable improved pedestrian detection on the roadside, since a larger field of view can be resolved. Even in rear view cameras, a larger field of view than that of Equation 1 can be imaged in a spatially resolved manner with a smaller image sensor. Since the resolution is not for all z. B. safety-relevant applications is sufficient, it may be advantageous to trigger a warning state based on detected objects in the area outside the maximum field of view.

This directed reflectance or mirror property can, for. B. by mirroring the at least one prism side surface or by gluing a mirrored element can be achieved. Furthermore, the directional reflectance can also be achieved by suitable surface structures, for. B. by repetitive wedge structures or diffraction gratings.

In a more advantageous embodiment, this reflection can be wavelength-dependent (dichroidic) or else selectively reflect or transmit between different polarization states. In a transmission through the mirror, the light is coupled out of the imaging beam path and does not reach the image sensor.

In particular, the filter array is adapted to a separation of the superimposed images.

Wherein the separation can be carried out depending on the wavelength depending on the angle of incidence or depending on the polarization state.

In a further advantageous embodiment, this property should at least partially be electronically controlled or regulated.

By means of a method according to the invention, it is possible to reconstruct the image data which are imaged directly on the image sensor and to reconstruct the image data which are reflected by the reflective element from the partially superimposed image data which the image sensor detects.

For this purpose, the superposed image area is determined and divided into directly and mirrored imaged electromagnetic radiation due to different dependence on the angle of incidence, the wavelength and / or the polarization state. To determine the total direct image data, the direct image data from the superimposed image area is extended around the direct area where no superimposition has occurred.

Further advantageous embodiments of the invention can be found in further subclaims.

With reference to the figures and embodiments, the invention will be explained in more detail below.

Show it:

1 a filter pixel arrangement in the form of a 2 × 2 matrix,

2 a large-scale arrangement of filter pixels,

3 an optical device with lens and image sensor according to the prior art,

4 through the optical device 4 not shown field angle ranges,

5 an optical device having a prism with a reflective side surface,

6 superimposed and not superimposed areas on the image sensor,

7 an optical device with a mirror as a reflective element,

8th A prism with a reflective side surface, where the reflection depends on the wavelength of the incident electromagnetic radiation.

1 illustrates the different properties of filter arrays of neighboring pixels.

The example shows four pixels ( 1 . 2 . 3 . 4 ), which are combined into a block. From this block, image processing can perform an identification of the electromagnetic radiation with respect to wavelength and / or polarization state.

The filter pixel matrix in 1 is structured as follows:
filter pixels 1 (top left) = 0 0;
filter pixels 2 (top right) = 0 1;
filter pixels 3 (bottom left) = 1 0;
filter pixels 4 (bottom right) = 1 1.

2 shows this filter pixel block ( 1 . 2 . 3 . 4 ) in the arrangement of filter pixels of the image sensor as a repetition sequence.

In 3 is an optical device or a camera system according to the prior art shown with the maximum field angle α, which results from the geometric distance of the sensitive image sensor surface according to equation 1. The image sensor width in this case corresponds to d3, the maximum field angle α corresponds to a1.

In 3 is a lens or a lens system ( 5 ) with behind or underlying aperture ( 8th ). Through this lens system ( 5 ) is electromagnetic radiation z. B. with an object-side focus on infinity in an image-side plane with the distance f (f = focal length) from the rear main point of the lens system ( 5 ). In this image plane is the image sensor ( 5 ).

Now consider the center beams (chief rays) of the imaging beam paths from the leftmost and the rightmost pixels of the image sensor ( 6 ) results in an object-side distance d2 from the entrance pupil (denoted by + in 3 ) a maximum field of view of d1. This maximum field of view is defined by the field of view angle α = a1 given by Equation 1.

For many applications, it is desirable or necessary that the maximum field of view associated with an image sensor ( 6 ) with a given optical system (lens ( 5 ), Cover ( 8th ) etc.) can be detected is extended.

Electromagnetic radiation emanating from object points or surfaces having a greater longitudinal extent than the width d1 at a distance d2, or electromagnetic radiation incident to the optical system ( 5 . 8th ) is emitted at an angle which is greater than a1, can not with the image sensor ( 6 ) are spatially resolved. This radiation is z. From one behind the image sensor ( 6 ) board (not shown) scattered to one part and directly absorbed to another part.

This situation is in 4 shown. For the sake of clarity, only the center beams (main beams or chiefrays) are off 3 shown.

The maximum field of view on the image sensor ( 6 ) can be spatially resolved is given by the object-side distance d1. Electromagnetic radiation radiated from an object point or object area cutout located in a distance range d5 outside the maximum field of view (d1) is measured away from the image sensor (FIG. 6 ) (distance range d4 next to the image sensor) and thus can no longer be detected by the image sensor ( 6 ) are spatially resolved. This applies to all object points which are located in an object-side distance which is greater than the maximum field of view d1 at an associated distance d2 or in which the angle a2 is greater than the maximum field angle a1.

How a reflection according to the invention can be implemented is explained in two further figures:
5 shows as a first embodiment, a prism ( 9 ) having at least one side surface which receives by coating or joining technique with a further element the properties of the directed reflectance described below.

7 shows as a second embodiment, a mirror ( 10 ), the image sensor ( 6 ) also causes a directed reflectance.

In 5 is the side surface of a prism ( 9 ) having the properties according to the invention for image superimposition in a region of the image sensor ( 6 ).

The orientation of the prism side surface at a 90 ° angle to the sensitive semiconductor surface is not mandatory. If a distortion of the image can be tolerated or even desired, this angle may also deviate accordingly.

Specially in 5 is passed through the prism body ( 9 ) used as a support for the surface with the directed reflectance ( 10 ), causes an axial offset in the imaging beam path. This must be in the camera structure by shifting the image sensor ( 6 ) are corrected by the distance d6 to continue to obtain a sharp image. The prism body ( 9 ) is preferably to be designed so that it geometrically with the sensitive semiconductor surface of the image sensor ( 6 ), ie has a length d3.

The prism body ( 9 ) can in a preferred embodiment only in a partial area (ie, partially) over the sensitive semiconductor surface of the image sensor ( 6 ) be applied.

Alternatively, a prism body ( 9 ) with at least two differently high subregions above the sensitive semiconductor surface of the image sensor ( 6 ) can be arranged.

Both embodiments in particular enable a camera-based rain sensing for a vehicle while observing the vehicle environment. On a portion of the image sensor ( 6 For this purpose, an optical proximity area (for example windscreen outer side for rain detection) and on a second subarea of the image sensor (FIG. 6 ) a long distance optical range (eg the area in front of a motor vehicle) is shown.

The superimposition of the image areas is in 6 exemplified horizontally. The image from the optical device without the reflective element is imaged directly on the regions i1 + i2 + i3. Ie. without reflection, the "ordinary" image comprises the regions i1 + i2 + i3 and the content fills the distance range d1 3 or 4 , The through the reflective element ( 10 ) additionally mirrored beam paths of electromagnetic radiation, which is radiated from field of view angles a1 to a2 (see 5 ie larger than the maximum viewing angle range according to equation 1) are superimposed on objects left to the optical axis in the image sensor area i3. If optional, a second reflective element on the left side surface of the prism 5 attached (in 5 not shown), objects from field of view angles a1 to a2 are imaged to the right of the optical axis in the image sensor area i1 (superimposed with the direct image in this area).

In 7 is a reflective surface in the form of a mirror ( 10 ), which is normal, ie at a right angle to the image sensor ( 6 ), and allows a superimposed image of object-side field of view angles a1 to a2 that are greater than the maximum field of view that results from Equation 1.

The orientation of the mirror ( 10 ) at a 90 ° angle to the sensitive semiconductor surface of the image sensor ( 6 ) is not mandatory. If distortion of the image can be tolerated or even desired, this angle may differ in both directions.

The advantage resulting from the invention is a larger field of view, which is geometrically on the sensitive semiconductor surface of the image sensor ( 6 ) can be resolved.

A differentiation possibility for separating the superimposed image areas (i1, i3) will be explained in more detail below.

In particular, the filter array is adapted to a separation of the superimposed images. The separation can be carried out depending on the wavelength, depending on the angle of incidence or depending on the polarization state.

Based on the wavelength-dependent filter function, the principle should be explained as an example:
As in 1 have shown the sensitive semiconductor portions of the image sensor ( 6 ) a superimposed plane with a filter array, which allows a distinction of the electromagnetic radiation based on a signal processing of the four pixels here together. In the following, this filter array is defined as a wavelength-dependent filter array.

The example shows four filter pixels ( 1 . 2 . 3 . 4 ), which are combined into a block. From this block, image processing can perform an identification of the electromagnetic radiation with respect to the wavelength.

For the filter pixel matrix off 1 For example, the following wavelength ranges are transmitted in each case:
filter pixels 1 (0 0): 300 nm to 430 nm;
filter pixels 2 (0 1): 400 nm to 560 nm;
filter pixels 3 (10): 540 nm to 720 nm;
filter pixels 4 (11): 700 nm to 1200 nm.

These values of the wavelength ranges are exemplary, the invention is not limited thereto.

In a superimposition of the viewing angle ranges by reflection (see. 5 and 7 ), the reflective surface ( 10 ) is advantageously designed as a wavelength-dependent mirror.

As a result, the desired wavelength ranges are mirrored, the "unwanted" wavelength ranges are coupled out or transmitted and do not come on the image sensor ( 6 ) to a spatially resolved image.

This is schematically in 8th shown:
As an example, only electromagnetic radiation in a first wavelength range λ1 = 700 nm to 1200 nm is mirrored. Electromagnetic radiation in a second wavelength range λ2, which corresponds to an area with wavelengths smaller or larger than λ1, is thus not mirrored, but transmitted and leaves the observed imaging system.

For signal processing this means: An overall image, which is transmitted by the image sensor ( 6 ) is subdivided into the four subpictures, which then exclusively from one of the following filter pixels ( 1 . 2 . 3 . 4 ) are: 0 0, 0 1, 1 0, 1 1.

For the separation of the superimposed partial areas in the picture (ie in 6 range i1 or i3), an average value (offset) from the signal values of the separated subimages 0 0, 0 1, 1 0 multiplied by a weighting function is subtracted from the subimage 11.

The sub-picture 1 1 then contains, except for residual errors, only the mirrored-in additional field angle range a1 to a2 as the signal value. The resolution of this figure is thus at most one quarter of the total resolution.

The subpictures 0 0, 0 1, 1 0 contain the information concerning the original image, d. H. without the (mirrored) electromagnetic radiation, which is outside the maximum viewing angle range a1. If necessary, the corresponding subimage 11 'without the mirrored portion can be interpolated from the other three subimages (comparable to the averaging described).

The ratio of signal quality of the mirrored image to residual errors of the original image in the superimposed area can be increased by a plane between mirror or reflective prism side surface (FIG. 10 ) and sensitive semiconductor surface of the image sensor ( 6 ) that reflects or transmits electromagnetic radiation depending on the angle of incidence. Since the mirrored additional field angle ranges a1 to a2 have greater angles of incidence than electromagnetic radiation, which is imaged at field angles smaller than a1, this can be used for differentiation.

This could be for a filter element z. B. Filter pixels ( 4 ) (ie 1 1) in the image sensor array an incident angle-dependent transmission are possible. Whereby z. B. is reflected for incident angle less than 10 ° and thus an image on the sensitive semiconductor surface of the image sensor ( 6 ) is suppressed. Thus, only electromagnetic radiation with angles of incidence greater than 10 ° reaches the sensitive semiconductor surface and thus preferably the mirrored additional field angle ranges a1 to a2.

The imaging quality of these additional field angles can be enhanced, in particular in this example, by an additional illumination of these object-side field angles in the wavelength range used. Here in the example, this would be a near-infrared illumination with a wavelength in the range of 700 nm to 1200 nm.

LIST OF REFERENCE NUMBERS

1

Filter pixel 0 0

2

Filter pixel 0 1

3

Filter pixel 1 0

4

Filter pixel 1 1

5

Lens or lens system

6

image sensor

7

Optical axis

8th

cover

9

prism

10

Reflective surface

+

entrance pupil

a1

maximum field of view angle

a2

Angle greater than a1

d1

object-side maximum horizontal field of view

d2

Object-side distance to the entrance pupil

d3

Width of the image sensor

d4

Distance range next to the image sensor

d5

object-side distance range outside of d1

d6

Displacement of the image sensor to correct the axial offset

i1, i3

Image sensor area with superposition of direct with mirrored image

i2

Image sensor area without overlay

λ1

first wavelength range

λ2

second wavelength range

λ1 + 2

Area comprising first and second wavelength range

Claims (15)

Optical device comprising an image sensor ( 6 ) a lens system ( 5 ) for projecting electromagnetic radiation from an object-side region up to a maximum field of view angle a1 to the optical axis ( 7 ) on the image sensor ( 6 ), characterized in that a reflective element ( 10 ) is arranged in the optical device such that a reflection of beam paths with field angles greater than the maximum field of view angle a1 on the image sensor ( 6 ) he follows. Optical device according to claim 1, wherein the reflective element ( 10 ) electromagnetic radiation as a function of the angle of incidence, reflected by the wavelength and / or the polarization state. Optical device according to claim 1 or 2 comprising a matrix-like arrangement of filter pixels ( 1 . 2 . 3 . 4 ) for pixels of the image sensor ( 6 ), causing electromagnetic radiation incident on a pixel of the image sensor ( 6 ), as a function of the angle of incidence, of the wavelength and / or of the polarization state through the filter pixels (FIG. 1 . 2 . 3 . 4 ) is filtered. An optical device according to claim 3, wherein the device consists of a 2 × 2 filter pixel matrix ( 1 . 2 . 3 . 4 ) is constructed. Optical device according to claim 4, wherein the 2 × 2 filter pixel matrix ( 1 . 2 . 3 . 4 ) a first filter pixel ( 1 ) with a transmission in the blue, a second filter pixel ( 2 ) with a transmission in the green, a third filter pixel ( 3 ) with a transmission in the red and a fourth filter pixel ( 4 ) with a transmission in the near-infrared wavelength range. Optical device according to claim 4, wherein the 2 × 2 pixel matrix ( 1 . 2 . 3 . 4 ) a first filter pixel ( 1 ) with a transmission in the range of 300 nm to 430 nm, a second filter pixel ( 2 ) having a transmission in the range of 400 nm to 560 nm, a third filter pixel ( 3 ) with a transmission in the range from 540 nm to 720 nm and a fourth filter pixel ( 4 ) having a transmittance in the range of 700 nm to 1200 nm. Optical device according to one of the preceding claims, wherein the reflective element ( 10 ) reflects only electromagnetic radiation having a wavelength in the infrared or near-infrared wavelength range. Optical device according to one of the preceding claims comprising a prism ( 9 ) having at least one side surface, wherein the reflective element ( 10 ) by a reflective coating of the side surface or by attaching a reflective surface to the side surface of the prism ( 9 ) is trained. Optical device according to one of the preceding claims, wherein the reflective element ( 10 ) at right angles to the optical axis ( 7 ) is arranged. Optical device according to one of the preceding claims, wherein the reflective element ( 10 ) is formed by surface structures, in particular by repeated wedge structures or diffraction gratings. Optical device according to one of the preceding claims comprising a reflective element ( 10 ), which is electronically controlled or regulated. Optical device according to one of the preceding claims comprising a light source for at least partially targeted illumination of the object-side field angle ranges with field angles greater than the maximum field of view angle a1. An optical device according to claim 11, wherein the light source generates electromagnetic radiation having a wavelength in the infrared or near-infrared wavelength range. Method for evaluating image data recorded with an optical device according to one of the preceding claims, having the following steps: • splitting the image data into a direct (i2) and a superposed (i1, i3) region • splitting the superimposed image region (i1, i3 ) due to different dependence on the angle of incidence, of the wavelength and / or of the state of polarization between direct and mirrored imaging of electromagnetic radiation into a first subimage for the pixel of the image sensor ( 6 ) with a selected filter pixel ( 4 ) are taken into account, on which in addition to the direct also mirrored electromagnetic radiation is imaged, and at least a second sub-image for the pixel of the image sensor ( 6 ) with one or more other filter pixels ( 1 . 2 . 3 ) are considered, on which electromagnetic radiation is imaged only directly. • Determining the image data mirrored in the superimposed area (i1, i3) by subtracting a weighted average value of signal values from the at least second subimage from the signal value of the first subimage. • Determine the direct image data by extending the direct area (i2) around the second sub-image. The method of claim 14, wherein the direct image data is interpolated in the superimposed (i1, i3) region by completing missing signal values in the at least second subimage with the weighted average values from the second subimage.

Images