WO2022199416A1

WO2022199416A1 - Camera module, terminal device, and imaging method

Info

Publication number: WO2022199416A1
Application number: PCT/CN2022/080768
Authority: WO
Inventors: 刘军; 肖锡晟; 卢佐旻; 许靖; 章浩亮
Original assignee: 华为技术有限公司
Priority date: 2021-03-24
Filing date: 2022-03-14
Publication date: 2022-09-29
Also published as: CN115134480A

Abstract

A camera module, a terminal device, and an imaging method, capable of being applied to the fields of automatic driving, assisted driving, security monitoring, etc. The camera module comprises a first optical lens component, a light splitting component, and a first image sensing component and a second image sensing component having the same photosensitive area; the resolution of the first image sensing component is greater than the resolution of the second image sensing component; the first optical lens component can be used for receiving light from a target object; and the light splitting component is used for splitting the light propagating via the first optical lens component to obtain first light and second light, propagating the first light to the first image sensing component, and propagating the second light to the second image sensing component. For the camera module, on the basis of two image sensing components having the same photosensitive area and different resolutions, the dynamic range of the camera module can be improved. The method can be applied to the Internet of Vehicles, such as vehicle to everything V2X, inter-vehicle communication long term evolution technology LTE-V, and vehicle‑to‑vehicle V2V.

Description

A camera module, terminal equipment and imaging method

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese patent application filed on March 24, 2021 with the application number 202110311311.X and the application name "A camera module, terminal equipment and imaging method", the entire contents of which are approved by Reference is incorporated in this application.

technical field

The present application relates to the technical field of camera modules, and in particular, to a camera module, a terminal device and an imaging method.

Background technique

With the development of technology, more and more functions, such as camera functions, are integrated on various devices. Users have higher and higher requirements for camera functions, for example, users need to obtain higher quality images. The following is an example of a vehicle-mounted camera.

In-vehicle cameras play an increasingly important role in assisted driving and autonomous driving. The images captured by the on-board camera can reflect the environment around the vehicle, thus providing necessary information for safe driving. The dynamic range in the vehicle-mounted camera is a relatively important functional parameter. The dynamic range refers to the interval included in the brightness values of the brightest and darkest objects that can normally display details in the same picture captured by the vehicle-mounted camera. Therefore, how to obtain high-quality images with a wide dynamic range is a technical problem that needs to be solved urgently in the vehicle camera.

At present, most of the solutions are to improve the dynamic range of the image by optimizing the pixels included in the sensor in the vehicle camera, such as optimizing the exposure time of the pixels. However, since the optimization of pixels is limited, the improvement of the dynamic range of the camera module is also limited.

SUMMARY OF THE INVENTION

The present application provides a camera module, a terminal device and an imaging method, which are used to improve the dynamic range of the camera module.

In a first aspect, the present application provides a camera module, the camera module may include a first optical lens assembly, a light splitting assembly, a first image sensing assembly and a second image sensing assembly, the first image sensing assembly and the first image sensing assembly. The photosensitive areas of the two image sensing assemblies are the same, and the resolution of the first image sensing assembly is greater than the resolution of the second image sensing assembly. Wherein, the first optical lens assembly is used to receive light from the target object; the light splitting assembly is used to split the light propagating through the first optical lens assembly to obtain the first light and the second light, which are sent to the first image sensing assembly The first light is propagated, and the second light is propagated to the second image sensing component.

Based on the above solution, since the sensitivity and resolution are trade-offs when the photosensitive area of the image sensing assembly is fixed, the sensitivity of the first image sensing assembly is smaller than the sensitivity of the second image sensing assembly. In other words, the first image sensing component has high resolution and low sensitivity to light, is not easy to overexpose to bright light, can identify high-brightness light, and can be used for imaging of bright targets; the second image sensing component It has lower resolution and higher sensitivity to light, can identify low-brightness light, and can be used for imaging of dim targets. In this way, the camera module can image both dim targets and bright targets, thereby helping to improve the dynamic range of the camera module.

In a possible implementation manner, the camera module may further include a first polarizer located between the light splitting component and the first image sensing component.

Through the first polarizer, the first light rays parallel to the polarization direction of the first polarizer can pass through, and the intensity of the first light rays entering the first image sensing component can be further weakened, thereby further weakening the transmission of the first image. Overexposure of the sensor component to bright light, which in turn helps to further improve the dynamic range of the camera module.

Further, optionally, the polarization direction of the first polarizer is perpendicular to the main polarization direction of the glare propagating through the first optical lens assembly.

By introducing a first polarizer in the optical path of the high-resolution first image sensing component, and designing the polarization direction of the first polarizer to be perpendicular to the main polarization direction of the received glare, it is possible to eliminate or weaken the first polarizer entering the high-resolution first image sensor. The glare of an image sensing component, so that the camera module can obtain a glare-free and high dynamic range image.

In a possible implementation manner, the ratio of the intensity of the first light ray to the second light ray is less than 1. In this way, the intensity of the first light entering the high-resolution first image sensing component can be further reduced, and the intensity of the second light entering the low-resolution second image sensing component can be increased, thereby helping Further improve the dynamic range of the camera module.

In a possible implementation manner, the first image sensing component is used for photoelectric conversion of the received first light to obtain information of the first image; the second image sensing component is used for the received second light Photoelectric conversion is performed to obtain information of the second image; the information of the first image and the information of the second image are used to form an image of the target object.

Further, optionally, the camera module further includes a first processing component, the first processing component can receive the information of the first image from the first image sensing component and receive the second image from the second image sensing component information, and generate an image of the target object according to the information of the first image and the information of the second image.

Specifically, the first processing component is configured to upsample the information of the second image to obtain the information of the third image, and the resolution of the third image corresponding to the information of the third image is the first image corresponding to the information of the first image. The resolutions of the images are the same; the information of the first image and the information of the third image are fused to obtain the image of the target object.

Through the fusion of the information of the first image and the information of the second image by the first processing component, the obtained image of the target object has higher quality.

In a second aspect, the present application provides a terminal device, and the terminal device can be any camera module in the first aspect or the first aspect.

Exemplarily, the terminal device may be, for example, a smartphone, a vehicle, a smart home device, a smart manufacturing device, a robot, a drone, a surveying and mapping device, or an intelligent transportation device.

In a third aspect, the present application provides an imaging method, the imaging method can be applied to a camera module, the camera module can include a first image sensing component and a second image sensing component, the first image sensing component and the first image sensing component The photosensitive areas of the two image sensing assemblies are the same, and the resolution of the first image sensing assembly is greater than the resolution of the second image sensing assembly. The method includes receiving light from a target object; splitting the light from the target object to obtain a first light and a second light, and propagating the first light to the first image sensing component, and propagating the second light to the second image sensing component light.

In a possible implementation manner, the camera module further includes a first polarizer located between the light splitting component and the first image sensing component.

In a possible implementation manner, the polarization direction of the first polarizer is perpendicular to the main polarization direction of the glare light propagating through the first optical lens assembly.

In a possible implementation manner, the ratio of the intensity of the first light ray to the second light ray is less than 1.

In a possible implementation manner, photoelectric conversion is performed on the first light to obtain the information of the first image; photoelectric conversion is performed on the second light to obtain the information of the second image; according to the information of the first image and the information of the second image information to generate an image of the target object.

Further, optionally, the information of the second image can be upsampled to obtain the information of the third image, and the information of the first image and the information of the third image can be fused to obtain the image of the target object, wherein the information of the third image is The corresponding third image has the same resolution as the first image corresponding to the information of the first image.

In a possible implementation manner, the camera module applicable to the third aspect may be the first aspect or any camera module in the first aspect.

For the technical effects that can be achieved in any one of the second aspect and the third aspect, reference may be made to the description of the beneficial effects in the first aspect, which will not be repeated here.

In a fourth aspect, the present application provides a camera module including a second optical lens assembly, a third optical lens assembly, a first image sensing assembly and a second image sensing assembly, the first image sensing assembly and the second image sensing assembly The photosensitive areas of the sensing assemblies are the same, and the resolution of the first image sensing assembly is greater than the resolution of the second image sensing assembly; the second optical lens assembly is used to receive light from the target object and transmit light to the first image sensing assembly. Propagating the third light; the third optical lens assembly is used for receiving the light from the target object, and propagating the fourth light to the second image sensing assembly.

Based on this solution, through the second optical lens assembly and the third optical lens assembly, the camera module can realize dual-channel imaging, and the second optical lens assembly can transmit the third light from the target object to the first image sensing assembly, The third optical lens assembly can transmit the fourth light from the target object to the second image sensing assembly. Since the sensitivity and resolution are trade-offs when the photosensitive area of the image sensing assembly is fixed, the sensitivity of the first image sensing assembly is smaller than the sensitivity of the second image sensing assembly. That is to say, the first image sensing component has high resolution and low sensitivity to light, is not easily overexposed to bright light, can identify high-brightness light, and can be used for imaging of bright targets; The sensor component has lower resolution and higher sensitivity to light, can recognize low-brightness light, and can be used for imaging of dim targets. In this way, the camera module can image both dim targets and bright targets, thereby helping to improve the dynamic range of the camera module.

In a possible implementation manner, the camera module further includes a second polarizer located between the second optical lens assembly and the first image sensing assembly.

Through the second polarizer, the third light rays parallel to the polarization direction of the second polarizer can pass through, and the intensity of the third light rays entering the first image sensing component can be further weakened, thereby further weakening the transmission of the first image. Overexposure of the sensor component to bright light, which in turn helps to further improve the dynamic range of the camera module.

Further, optionally, the polarization direction of the second polarizer is perpendicular to the main polarization direction of the glare light propagating through the second optical lens assembly.

By introducing a second polarizer in the optical path of the high-resolution first image sensing component, and designing the polarization direction of the second polarizer to be perpendicular to the main polarization direction of the received glare, it is possible to eliminate or weaken the entry into the high-resolution first image sensor. The glare of an image sensing component, so that the camera module can obtain a glare-free and high dynamic range image.

In a possible implementation manner, the aperture number of the second optical lens assembly is greater than the aperture number of the third optical lens assembly.

The amount of light passing through different channels (that is, the intensity of light) can be controlled by the aperture of the corresponding optical lens assembly. The intensity of the third light of an image sensing element is compared with the intensity of the fourth light entering the second image sensing element with low resolution, that is, the second optical lens element with a large aperture is matched with the first image with high resolution In the sensing assembly, the third optical lens assembly with a small aperture number cooperates with the second image sensing assembly with low resolution, thereby helping to further improve the dynamic range of the camera module.

In a possible implementation manner, the second optical lens assembly and the third optical lens assembly have the same focal length. In this way, the third light passing through the second optical lens assembly and the fourth light passing through the third optical lens assembly can be converged on the same plane, so that the first image sensing assembly and the second image sensing assembly can be arranged in the same plane on the substrate, which helps to simplify the assembly of the camera module.

In a possible implementation manner, the first image sensing component is used to perform photoelectric conversion on the received third light to obtain information of the fourth image; the second image sensing component is used for the received fourth light Photoelectric conversion is performed to obtain information of the fifth image; wherein, the information of the fourth image and the information of the fifth image are used to form an image of the target object.

Further, optionally, the camera module further includes a second processing component for receiving information of the fourth image from the first image sensing component and receiving information from the fifth image from the second image sensing component, and According to the information of the fourth image and the information of the fifth image, an image of the target object is obtained.

Specifically, the second processing component is configured to upsample the information of the fifth image to obtain the information of the sixth image, and the resolution of the sixth image corresponding to the information of the sixth image is the fourth image corresponding to the information of the fourth image The resolution is the same, and the information of the fourth image and the information of the sixth image are fused to obtain the image of the target object.

In a fifth aspect, the present application provides a terminal device, and the terminal device can be any camera module in the fourth aspect or the fourth aspect.

In a sixth aspect, the present application provides an imaging method, which can be applied to a camera module, and the camera module can include a first image sensing assembly and a second image sensing assembly, the first image sensing assembly and the second image sensing assembly. The photosensitive areas of the image sensing assemblies are the same, and the resolution of the first image sensing assembly is greater than the resolution of the second image sensing assembly. The method includes receiving light from a target object, and propagating a third light to a first image sensing assembly and a fourth light to a second image sensing assembly.

In a possible implementation manner, the second optical lens assembly and the third optical lens assembly have the same focal length.

In a possible implementation manner, photoelectric conversion may be performed on the received third light to obtain information of the fourth image; photoelectric conversion may be performed on the received fourth light to obtain information of the fifth image; The information of the image and the information of the fifth image are used to obtain the image of the target object.

Specifically, the information of the fifth image is up-sampled to obtain the information of the sixth image, and the information of the fourth image and the information of the sixth image are fused to obtain the image of the target object, wherein the information of the sixth image corresponds to the sixth image. The resolution of the image is the same as the resolution of the fourth image corresponding to the information of the fourth image.

In a possible implementation manner, the camera module applied in the sixth aspect may be any one of the fourth aspect or the fourth aspect.

For the technical effects that can be achieved in any one of the fifth aspect and the sixth aspect, reference may be made to the description of the beneficial effects in the fourth aspect, which will not be repeated here.

In a seventh aspect, the present application provides a computer-readable storage medium in which a computer program or instruction is stored, and when the computer program or instruction is executed by the camera module, the camera module is made to perform the above-mentioned third aspect Or the method in any possible implementation manner of the third aspect, or the camera module is made to execute the above sixth aspect or the method in any possible implementation manner of the sixth aspect.

Description of drawings

1a is a schematic diagram of the relationship between the size and resolution of a pixel provided by the application;

1b is a schematic diagram of the relationship between the size and resolution of another pixel provided by the application;

Fig. 1c is the schematic diagram of the polarization state of a kind of natural light provided by the application;

1d is a schematic diagram of the polarization state of a linearly polarized light provided by the application;

1e is a schematic diagram of the polarization state of another linearly polarized light provided by the application;

Figure 1f is a schematic diagram of the polarization state of a partially polarized light provided by the application;

FIG. 1g is a schematic diagram of the principle of a kind of image fusion provided by the application;

Fig. 2 is the principle schematic diagram of a kind of polarizer anti-glare provided by the application;

3a is a schematic diagram of a possible application scenario of a camera module provided by the application;

3b is a schematic diagram of another possible application scenario of the camera module provided by the application;

4 is a schematic structural diagram of a camera module provided by the application;

5a is a schematic structural diagram of a first optical lens assembly provided by the application;

5b is a schematic structural diagram of another first optical lens assembly provided by the application;

6 is a schematic diagram of the spectroscopic principle of a spectroscopic component provided by the present application;

7 is a schematic diagram of the relationship between a first image sensing assembly and a second image sensing assembly provided by the application;

8 is a schematic diagram of a process of fusion of a first image and a second image provided by the present application;

9 is a schematic structural diagram of another camera module provided by the application;

10 is a schematic structural diagram of another camera module provided by the application;

11 is a schematic structural diagram of another camera module provided by the application;

12 is a schematic structural diagram of a terminal device provided by the application;

13 is a schematic flowchart of an imaging method provided by the application;

FIG. 14 is a schematic flowchart of another imaging method provided by the present application.

Detailed ways

The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Hereinafter, some terms used in this application will be explained. It should be noted that these explanations are for the convenience of those skilled in the art to understand, and do not constitute a limitation on the protection scope required by the present application.

1. Dazzle

Glare refers to the visual conditions that cause visual discomfort and reduce the visibility of objects due to the extreme brightness contrast in space or time due to unsuitable brightness distribution in the field of view. A bright sensation in the visual field that the human eye cannot adapt to, which may cause disgust, discomfort or even loss of vision. Excessive brightness in a certain part of the field of view or excessive brightness changes before and after. For example, when a car is driving on the road, the road that has been sprinkled by rain is a good mirror surface, and strong light sources (such as the sun) are reflected by the mirror and enter the vehicle camera, which will cause strong glare; In the direction of , due to the existence of temperature gradient, the refractive index is lower as it is closer to the road surface, and the light incident on the road surface will also bend. At this time, the road surface is equivalent to a good mirror surface, and the strong light source is reflected by the mirror surface and enters the vehicle camera to generate glare.

2. Dynamic range

The dynamic range is an important parameter of the camera module, which refers to the interval included in the brightness values of the brightest and darkest objects that can normally display details in the same picture captured by the camera module. The greater the dynamic range, the greater the degree to which objects that are too bright or too dark can be displayed properly in the same frame.

3. Upsampling

Upsampling can also be called image upscaling or image interpolation, and the main purpose is to upscale the image so that a higher resolution image can be obtained.

Upsampling principle: Image enlargement usually adopts the interpolation method, that is, on the basis of the original image pixels, a suitable interpolation algorithm is used to insert new elements between the pixels. The interpolation algorithm may be traditional interpolation, edge image-based interpolation, or region-based image interpolation, etc., which is not limited in this application.

4. Pixels

A pixel may refer to the smallest unit that constitutes an imaging area of an image sensor. Among them, the size of the pixel refers to the physical size of the pixel, that is, the distance between the centers of adjacent pixels.

5. Resolution

Resolution can refer to the maximum number of pixels (ie, photosensitive cells) available for imaging on an image sensor. It is usually measured by the product of the number of horizontal pixels and the number of vertical pixels, that is, resolution=number of horizontal pixels×number of vertical pixels.

It should be noted that under the same photosensitive area (or called the same target surface), the resolution and the size of the pixel are trade-offs. Referring to Figure 1a and Figure 1b, the relationship between pixel size and resolution under the same photosensitive area. The size of the pixel in FIG. 1a is a and the resolution is 4×4; the size of the pixel in FIG. 1b is a/2 and the resolution is 8×8. It can be determined from Fig. 1a and Fig. 1b that the smaller the size of the pixel, the higher the resolution; the larger the size of the pixel, the lower the resolution.

6. Minimum Illumination

Minimum illumination refers to the sensitivity of the image sensor to ambient light, or the darkest light required by the image sensor for normal imaging.

7. Aperture

The aperture can be used to control the amount of light entering the optical lens, that is, the aperture is used to determine the amount of light entering the optical lens. The size of the aperture is usually expressed by the F number, denoted as F/, and the F number is also called the aperture number. A large aperture optical lens has a small F-number, that is, a small aperture; an optical lens with a small aperture has a large F-number, that is, a large aperture.

Under the condition that the shutter remains unchanged, the smaller the F-number, the larger the aperture, the more light entering, and the brighter the picture; the larger the F-number, the smaller the aperture, the less light entering, and the picture is darker.

Eight, polarizer

Polarizers, also known as polarizers, are a type of optical filter. Polarizers are used to absorb or reflect light in one polarization direction and transmit light in another orthogonal polarization direction. The transmittance of light is directly related to its polarization state. Polarizers are generally classified into absorbing polarizers and reflective polarizers (RP). Absorptive polarizers strongly absorb one of the orthogonally polarized components of incident linearly polarized light, while the other component is less absorbed. Reflective polarizers can transmit linearly polarized light in a certain direction and can reflect light whose polarization direction is perpendicular to the transmitted direction. The absorbing polarizer may be, for example, a dichroic polarizer, and the reflective polarizer may be, for example, a polarizing beam splitter utilizing birefringence.

For natural light (see Figure 1c), after incident on the polarizer, the outgoing light becomes linearly polarized light, and the energy of the outgoing light becomes 50% of the energy of the incident light. For linearly polarized light (see Figure 1d or Figure 1e), after entering the polarizer, the outgoing light is still linearly polarized light, and the energy of the outgoing light becomes I ₀ ×cosθ^2, where I ₀ represents the incident linearly polarized light. Energy, θ represents the angle between the polarization direction of the incident linearly polarized light and the polarization direction of the linear polarizer. For partially polarized light (see Figure 1f), after entering the polarizer, the outgoing light becomes linearly polarized light, and the energy of the outgoing light attenuates and changes periodically with the polarization angle.

9. Polarization direction

The polarization direction is also referred to as the polarization direction or the polarization direction. This is because there is a certain characteristic direction in the polarizer, called the polarization direction. The polarizer only allows light parallel to the polarization direction to pass through, while absorbing or reflecting light perpendicular to the direction.

10. Image fusion

Image fusion is an image processing technology, which refers to the image data collected by multiple source channels about the same target through image processing and specific algorithm calculation, etc., to maximize the extraction of favorable information in each channel, and finally to synthesize high quality. (eg brightness, sharpness, color), the fused image has a higher resolution than the original image.

Please refer to FIG. 1g , which is a schematic diagram of the principle of image fusion provided by the present application. Since image fusion can utilize the spatial-temporal correlation and information complementarity of two (or multiple) images, the fused new image has a more comprehensive and clear description of the scene, which is more beneficial to the detection device identification and detection. It should be noted that, image fusion usually needs to ensure that each image to be fused has been registered and the pixel bit width is consistent.

Some terms involved in the present application are introduced above, and the technical features involved in the present application are introduced below. It should be noted that these explanations are for the convenience of those skilled in the art to understand, and do not constitute a limitation on the protection scope required by the present application.

As shown in FIG. 2 , a schematic diagram of the principle of anti-glare of a polarizer provided by the present application. When the incident angle θ is the Brewster angle, the reflected light is linearly polarized light, and the transmitted light is approximately natural light. When the incident angle θ is a non-Brewster angle incident, both the reflected light and the transmitted light are partially polarized light. Among them, Brewster's angle=arctan(n2/n1), wherein, n1 represents the refractive index of the medium where the incident light is located, and n2 represents the refractive index of the medium where the refracted light is located. According to the principle of Fresnel reflection, the glare reflected by the road surface is partially polarized light or linearly polarized light, and the reflected glare can be weakened or eliminated by adjusting the polarization direction of the polarizer. It should be understood that when the glare is partially polarized light, the angle between the main polarization direction (or called the long axis of polarization) of the glare and the road surface can be determined.

Based on the above content, possible application scenarios of the camera module in this application are given below. Exemplarily, the camera module can be installed on a vehicle (eg, an unmanned vehicle, a smart vehicle, an electric vehicle, a digital vehicle, etc.) as a vehicle-mounted camera, as shown in FIG. 3a. The vehicle-mounted camera can obtain measurement information such as the distance of surrounding objects in real time or periodically, so as to provide necessary information for operations such as lane correction, vehicle distance maintenance, and reversing. Because the vehicle camera can realize: a) target recognition and classification, such as various lane line recognition, traffic light recognition and traffic sign recognition, etc.; area), mainly for vehicles, ordinary road edges, side stone edges, boundaries without visible obstacles, unknown boundaries, etc.; c) The ability to detect laterally moving targets, such as pedestrians and vehicles crossing the intersection d) Localization and map creation, such as localization and map creation based on simultaneous visual localization and mapping (SLAM) technology. Therefore, in-vehicle cameras have been widely used in unmanned, autonomous, assisted driving, intelligent driving, connected vehicles, security monitoring, surveying and mapping and other fields. It should be understood that the camera module can be combined with an advanced driving assistant system (ADAS).

It should be noted that the above application scenarios are just examples, and the camera module provided in this application can also be applied in various other scenarios, and is not limited to the scenarios exemplified above. For example, the camera module can also be applied to terminal equipment or set in components of the terminal equipment. Transport vehicle (automated guided vehicle, AGV) or unmanned transport vehicle, etc.). For another example, the camera module can also be installed on the drone as an onboard camera. For another example, the camera module can also be installed on roadside traffic equipment (such as a roadside unit (RSU)) as a roadside traffic camera, as shown in Figure 3b, so as to realize intelligent vehicle-road collaboration.

Among them, the dynamic range is a more important functional parameter of the camera module. At present, the dynamic range of the camera module can only reach 80-120 decibels (dB), but the dynamic range of the target in nature can reach 180dB. Therefore, the dynamic range of the camera module needs to be improved. At present, the dynamic range of the camera module is mainly improved by optimizing the pixels included in the image sensor in the camera module. However, since the optimization of pixels is limited, the improvement of the dynamic range of the camera module is also limited. Therefore, how to obtain high-quality images with a wide dynamic range is an urgent technical problem to be solved by the camera module.

In view of this, the present application provides a camera module, which can obtain a wider dynamic range.

Based on the above content, the camera module proposed in the present application will be described in detail below with reference to FIG. 4 to FIG. 11 .

Example 1

As shown in FIG. 4 , it is a schematic structural diagram of a camera module provided by the present application. The camera module may include a first optical lens assembly 401 , a light splitting assembly 402 , a first image sensing assembly 403 and a second image sensing assembly 404 . With the same area, the resolution of the first image sensing assembly 403 is greater than that of the second image sensing assembly 404; the first optical lens assembly 401 is used to receive the light from the target object; the light splitting assembly 402 is used to The light transmitted by the lens assembly 401 is split to obtain the first light and the second light, and the first light is transmitted to the first image sensing element 403 and the second light is transmitted to the second image sensing element 404 .

Based on the camera module, when the photosensitive area of the image sensing assembly is fixed, the sensitivity and resolution are trade-offs. Therefore, the sensitivity of the first image sensing assembly is smaller than the sensitivity of the second image sensing assembly. That is to say, the first image sensing component has high resolution and low sensitivity to light, is not easily overexposed to bright light, can identify high-brightness light, and can be used for imaging of bright targets; The sensor component has lower resolution and higher sensitivity to light, can recognize low-brightness light, and can be used for imaging of dim targets. In this way, the camera module can image both dim targets and bright targets, thereby helping to improve the dynamic range of the camera module.

It should be understood that the camera module is equivalent to the second image sensing component sacrificing the resolution of a part of the image in exchange for an increase in sensitivity, so as to improve the dynamic range of the camera module.

In a possible implementation manner, the information carried by the light propagating through the first optical lens assembly is the same as the information carried by the light entering the first optical lens assembly (ie, the light from the target object).

It should be noted that the target object includes but is not limited to a single object. For example, when photographing a person, the target object includes the person and the scene around the person, that is, the scene around the person is also a part of the target object. It can also be understood that all objects within the field of view of the first optical lens assembly can be referred to as target objects.

The respective functional components and structures shown in FIG. 4 are introduced and described below to give an exemplary specific implementation solution. For the convenience of description, the following first optical lens assembly, light splitting assembly, first image sensing assembly and second image sensing assembly are not marked.

1. The first optical lens assembly

In a possible implementation manner, the first optical lens assembly can receive light from the target object, and by changing the propagation direction of the light from the target object, the light from the target object can enter the camera module as much as possible. Further, optionally, the first optical lens assembly also propagates glare (such as the glare reflected from the road surface) into the camera module.

In a possible implementation manner, the first optical lens assembly may be composed of at least one optical lens. As an example, FIG. 5a shows a schematic structural diagram of a first optical lens assembly. The first optical lens assembly includes seven optical lenses as an example. The light from the target object can be transmitted into the camera module through the first optical lens assembly as much as possible, and further, external glare can also be transmitted into the camera module through the first optical lens assembly.

As yet another example, FIG. 5b shows a schematic structural diagram of another first optical lens assembly. The first optical lens assembly may include six optical lenses.

It should be understood that the structure of the first optical lens assembly shown in FIG. 5a or 5b above is only an example, and the first optical lens assembly in the present application may have more or less optical lenses than those shown in FIG. Figure 5b More or fewer optical lenses. Wherein, the optical lens can be any one of a convex lens (such as a biconvex lens, a plano-convex lens, or a convex-concave lens) or a concave lens (such as a biconcave lens, a plano-concave lens, or a meniscus lens), or a combination of a convex lens and a concave lens, which is not covered in this application. Do limit.

In a possible implementation manner, in order to suppress temperature drift, the material of at least one optical lens in the first optical lens assembly is glass.

Further, optionally, in order to minimize the height of the camera module, the optical lenses in the first optical lens assembly can be cut in the height direction of the camera module (see FIG. 5a above).

2. Spectral components

In a possible implementation manner, the light splitting component may be configured to perform light splitting (eg, split into two) light transmitted by the first optical lens component to obtain the first light ray and the second light ray. Exemplarily, the light splitting component may divide the intensity (or called partial energy or called partial amplitude) of the light transmitted from the first optical lens component to obtain the first light ray and the second light ray. It should be understood that the information carried by the first light and the second light is the same, the information carried by the first light is the same as the information carried by the light transmitted by the first optical lens assembly, and the information carried by the second light is carried by the first optical lens assembly. The information carried by the light is the same. Wherein, the sum of the intensity of the first light and the intensity of the second light is equal to or approximately equal to the intensity of the light transmitted by the first optical lens assembly.

Further, optionally, the ratio of the intensities of the first light and the second light may be less than 1. For example, the ratio of the intensity of the first light and the second light may be 2:8; for another example, the ratio of the intensity of the first light to the second light is 1:9; for another example, the intensity of the first light and the second light The intensity ratio is 4:6. It can also be understood that the intensity of the first light received by the first image sensing assembly is smaller than the intensity of the second light received by the second image sensing assembly. In this way, the intensity of the first light received by the first image sensing component with high resolution and low sensitivity is smaller than the intensity of the second light received by the second image sensing component with low resolution and high sensitivity. The dynamic range of the camera module. It should be understood that the ratio of the intensities of the first light and the second light may also be equal to 1, or greater than 1.

It should be noted that the ratio of the intensities of the first light and the second light can be designed according to actual requirements, which is not limited in this application.

In a possible implementation manner, the light-splitting component may be, for example, a beam splitter (beam splitter, BS) or a light-splitting plate. The beam-splitting prism is formed by coating one or more thin films (ie, beam-splitting film) on the surface of the prism, and the beam-splitting plate is formed by coating one or more layers of thin films (ie, beam-splitting film) on one surface of the glass plate. The light splitting prism and the light splitting plate both use the films to have different transmittances and reflectances of the incident light, so as to realize the light splitting of the light transmitted by the first optical lens assembly.

As shown in FIG. 6 , a schematic diagram of a spectroscopic principle of a spectroscopic component provided by the present application. The beam splitting prism can divide the light transmitted by the first optical lens assembly into two to obtain the first light and the second light. It can also be understood that after the light propagating from the first optical lens assembly passes through the light splitting assembly, part of it is transmitted (the first light) to the first image sensor assembly, and the other part is reflected (the second light) to the second image sensor. components. Exemplarily, the ratio of the intensities of the first light and the second light may be determined by the reflectance and transmittance of the plated beam splitting film.

3. The First Image Sensing Assembly and the Second Image Sensing Assembly

In a possible implementation manner, the first image sensing component may perform photoelectric conversion on the received first light to obtain information of the first image; the second image sensing component may perform photoelectric conversion on the second light to obtain information of the second image.

Here, the first image sensing assembly and the second image sensing assembly use non-identical architectures. Specifically, the photosensitive areas of the first image sensing assembly and the second image sensing assembly are the same, and the resolution of the first image sensing assembly is greater than the resolution of the second image sensing assembly. Further, optionally, the first image sensing component includes a first pixel, the second image sensing component includes a second pixel, and the size of the first pixel is smaller than the size of the second pixel. It should be understood that the larger the size of the pixel, the higher the sensitivity of the image sensing assembly; the smaller the size of the pixel, the lower the sensitivity of the image sensing assembly, therefore, the sensitivity to light of the first image sensing assembly is lower than that of the second image sensing assembly The sensitivity of the image sensing component to light.

As shown in FIG. 7 , a schematic diagram of the relationship between a first image sensing assembly and a second image sensing assembly provided by the present application. (a) in FIG. 7 represents the first image sensing assembly, and (b) in FIG. 7 represents the second image sensing assembly. The minimum repeatable units of the first image sensing assembly and the second image sensing assembly are both RGGB, R represents a pixel for receiving red (red) light, G represents a pixel for receiving green (green) light, and B represents a pixel for receiving blue (blue) light. The photosensitive areas of the first image sensing assembly and the second image sensing assembly are the same, and the size of the first pixel included in the first image sensing assembly is twice the size of the second pixel included in the second image sensing assembly. Therefore, , the resolution of the first image sensing assembly is 4 times that of the second image sensing assembly. Also, the sensitivity of the first image sensing assembly is 1/4 of the sensitivity of the second image sensing assembly. That is to say, the first image sensing component has the characteristics of high resolution, small pixels and low sensitivity, and can be used for imaging of bright targets; the second image sensing component has the characteristics of low resolution, large pixels and high sensitivity, and can be used for imaging of dim targets .

It should be noted that the resolution and the minimum repeatable unit of the first image sensing assembly and the second image sensing assembly shown in FIG. 7 are examples, and do not limit the present application. For example, the smallest repeatable unit of the first image sensing assembly and the second image sensing assembly may also be RYYB.

With reference to the above FIG. 7 , the resolution of the first image sensing component is 8M, the resolution of the second image sensing component is 2M as an example, and the ratio of the intensity of the first light to the second light is 2:8 as an example . Table 1 exemplarily shows the relationship between the improved dynamic range of the camera module based on the first embodiment and the dynamic range of the camera module based on the prior art.

Table 1 The relationship between the dynamic range of the camera module based on the first embodiment and the dynamic range of the camera module based on the prior art

It should be noted that, the field of view of the first optical lens assembly may include a bright target or a dim target, and the bright target and the dim target are collectively referred to as a target object. As can be seen from the above table 1, based on the camera module in the prior art, the energy of the incident light received by the image sensor from the bright target is Eh, the energy of the incident light from the dim target is E1, and the dynamic range is -10log. (Eh/El). Based on the camera module of the first embodiment, since the ratio of the intensity of the first light and the second light after the light splitting by the light splitting component is 2:8, the first image sensing component receives the incident light from the bright target The energy of 0.2 × Eh. In this way, the intensity of the first light entering the first image sensing assembly is reduced by 0.2 times, and therefore, the overexposure prevention capability of the first image sensing assembly for the bright target is improved by two times. The energy of the incident light received by the second image sensing assembly from the dim target is 0.8×El; and the size of the second pixel included in the second image sensing assembly is the size of the first pixel included in the first image sensing assembly Therefore, the sensitivity of the second image sensing assembly is 4 times that of the first image sensing assembly, so the low illumination capability of the second image sensing assembly for dim targets is improved by 4×0.8 = 3.2 times. Therefore, the dynamic range of the camera module can be improved by -10log(Eh/El)=10log16. In this way, it can be further demonstrated that the dynamic range of the camera module based on the first embodiment has been effectively improved. It should be understood that the energy of the incident light received by the second image sensing component from the bright target is 0.8×Eh, which may be overexposed, and the energy of the incident light received by the first image sensing component from the dim target is 0.2×El , may be too dark, and a high dynamic range can be achieved through the fusion of image information by the first processing component later. For details, please refer to the subsequent related descriptions, which will not be repeated here.

In a possible implementation manner, the first image sensing component may be a complementary metal-oxide semiconductor (CMOS) phototransistor, a charge-coupled device (CCD), and a photodetector (photon detector, PD), or high-speed photodiode. Among them, a CMOS phototransistor is a device or chip that converts optical signals into electrical signals based on complementary metal oxide semiconductor technology. The second image sensing component may also be a complementary metal-oxide semiconductor (CMOS) phototransistor, a charge-coupled device (CCD), a photon detector (PD), or High-speed photodiode.

It should be noted that the type of the first image sensing element may be the same as the type of the second image sensing element, for example, the first image sensing element and the second image sensing element may both be CMOS phototransistors. Alternatively, the type of the first image sensing element may be different from that of the second image sensing element, for example, the first image sensing element is a CMOS phototransistor, and the second image sensing element is a CCD.

In a possible implementation manner, the first image sensing component may be a first image sensor, and the second image sensing component may be a second image sensor. Exemplarily, the resolution range of the first image sensor may be [8 million pixels, 48 million pixels], and the resolution range of the second image sensor may be [8 million pixels, 48 million pixels]. The resolution is greater than the resolution of the second image sensor. Exemplarily, the resolution of the first image sensor may be 12 million pixels, 20 million pixels, or 48 million pixels, and the resolution of the second image sensor may be 8 million pixels. Of course, the resolution of the first image sensor may also be greater than 48 million pixels, for example, 52 million pixels, 60 million pixels, 72 million pixels, etc.; the resolution of the second image sensor may also be greater than 8 million pixels. It should be understood that any combination that can satisfy that the resolution of the first image sensing assembly is greater than that of the second image sensing assembly is acceptable.

The camera module in the first embodiment may further include a first polarizer. Further, optionally, the camera module may further include a first processing component. The first polarizer and the first processing component are respectively introduced below.

Fourth, the first polarizer

In a possible implementation, the first polarizer may be located between the light splitting component and the first image sensing component (see Figure 9 below). The first polarizer is used for allowing the light of the first light whose polarization state is parallel to the polarization direction of the first polarizer to pass through, and the first light passing through the first polarizer is converged on the first image sensing component. In this way, the intensity of the first light entering the first image sensing element can be further reduced, thereby further reducing the overexposure of the first image sensing element to bright light, thereby further improving the dynamic range of the camera module. It should be noted that the first light rays passing through the first polarizer are part of the first rays entering the first polarizer (ie, the part of the first rays parallel to the polarization direction of the first polarizer).

Further, optionally, the polarization direction of the first polarizer is perpendicular to the main polarization direction of the glare propagating through the first optical lens assembly. By introducing a first polarizer in the optical path of the high-resolution first image sensing component, and designing the polarization direction of the first polarizer to be perpendicular to the main polarization direction of the received glare, it is possible to eliminate or weaken the first polarizer entering the high-resolution first image sensor. The glare of an image sensing component (the specific principle can be combined with the polarization state of the glare and the description of FIG. 2 above), so that a glare-free and high dynamic range image can be obtained.

With reference to the above FIG. 7 , the resolution of the first image sensing component is 8M, the resolution of the second image sensing component is 2M as an example, and the ratio of the intensity of the first light to the second light is 2:8 as an example . When the camera module further includes a first polarizer, the relationship between the dynamic range of the camera module based on Embodiment 1 and the dynamic range of the camera module based on the prior art can be seen in Table 2 below.

Table 2 The improved dynamic range of the camera module based on the first embodiment and the dynamic range of the camera module based on the prior art

When the camera module further includes a first polarizer, in combination with the above Table 2, similar to the analysis in Table 1 above, the overexposure prevention capability of the first image sensing component for high-brightness targets is improved by 10 times, and the second image sensor The low-light capability of the component for dim targets is increased by 3.2 times, and the dynamic range of the camera module is 10log32.

In a possible implementation manner, improving the dynamic range of the camera module is related to the ratio of the intensities of the first light and the second light. When the camera module further includes the first polarizer, please refer to Table 3 for the relationship between the intensity ratio η of the first light and the second light and the effect of improving the dynamic range.

Table 3 The relationship between the ratio η of the intensity of the first light and the second light and the improvement effect of the dynamic range

ηn	1:91:9	2:82:8	3:73:7	4:64:6	5:55:5
动态范围的提升(倍)Dynamic range improvement (times)	20×3.6＝7220×3.6=72	10×3.2＝3210×3.2=32	6.7×2.8＝18.76.7×2.8=18.7	5×2.4＝125×2.4=12	4×2＝84×2=8
动态范围的提升(dB)Dynamic range improvement (dB)	18.518.5	1515	12.712.7	10.710.7	99

It should be understood that the relationship between the improvement (dB) of the dynamic range and the improvement (times) of the dynamic range satisfies: the improvement of the dynamic range (dB)=-10log (the improvement (times) of the dynamic range).

When the camera module includes the first polarizer, the first polarizer can not only eliminate or reduce glare, but also soften the first light received by the first image sensing component.

5. The first processing component

In a possible implementation manner, the first processing component may receive information of the first image from the first image sensing component and receive information of the second image from the second image sensing component, according to the information of the first image information and information of the second image to generate an image of the target object. Specifically, the first processing component may perform up-sampling on the information of the second image (for details, please refer to the foregoing related description) to obtain the information of the third image, and the information of the third image corresponding to the resolution of the third image is the same as that of the first image. The resolution of the first image corresponding to the information of the image is the same; the information of the first image and the information of the third image are fused to obtain the image of the target object, as shown in FIG. 8 . By fusing the information of the first image and the information of the second image, the obtained image of the target object has higher quality.

Further, optionally, the first processing component may further perform processing such as denoising, enhancement, segmentation and blurring on the image of the target obtained by fusion, so as to enrich the user experience.

In a possible implementation manner, the first processing component may be, for example, an application processor (application processor, AP), a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a digital Signal processor (digital signal processor, DSP) and so on.

Alternatively, the first processing component may be a central processing unit (CPU), other general-purpose processors, application specific integrated circuits (ASICs), field programmable gate arrays (field programmable gate arrays, FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

Alternatively, the first processing component may be a combination of any of a CPU, ASIC, FPGA, other programmable logic device, transistor logic device and any of an application processor, graphics processor, image signal processor, digital signal processor .

It should be noted that, the camera module of one of the above embodiments may further include an infrared (infrared radiation, IR) filter, and the IR filter may be located between the light splitting component and the first image sensing component, and/or , between the light splitting component and the second image sensing component. The IR filter can be used to block or absorb infrared light to prevent damage to the image sensing assembly; furthermore, the IR filter can be configured to have no effect on the focal length of the first optical lens assembly. Illustratively, the material of the IR filter may be glass or a glass-like resin, such as blue glass.

Based on the above content, a specific implementation manner of the camera module in the above-mentioned first embodiment is given below with reference to the specific hardware structure. In order to further understand the structure of the above-mentioned camera module and the formation process of the image of the target object.

As shown in FIG. 9 , it is a schematic structural diagram of another camera module provided by the present application. The camera module may include a first optical lens assembly, a light splitting assembly, a first image sensing assembly, a second image sensing assembly, a first polarizer located between the light splitting assembly and the first image sensing assembly, and a first process components. 5a as an example for the first optical lens assembly, a prism as an example for the spectroscopic component, the (a) in FIG. 7 for the first image sensing component as an example, and the second image sensing component for the Take (b) as an example.

For possible implementations of the first optical lens assembly, the light splitting assembly, the first image sensing assembly, the second image sensing assembly, the first polarizer, and the first processing assembly, reference may be made to the foregoing related descriptions, which will not be repeated here. Repeat.

Based on the camera module shown in FIG. 9 , the light from the target object passes through the first optical lens assembly and is transmitted to the light splitting assembly, and is divided into a first light ray and a second light ray by the light splitting assembly, and the first light ray propagates to the first image sensor. The second light is transmitted to the second image sensing assembly, the first image sensing assembly converts the first light into a first electrical signal (that is, the information of the first image), and the second image sensing assembly converts the second light Converted into a second electrical signal (that is, the information of the second image), the first electrical signal can be converted into a first digital image signal through an analog to digital converter (A/D), and the second electrical signal can be converted into a first digital image signal. After A/D, it becomes a second digital image signal. Both the first digital image signal and the second digital image signal are transmitted to the ISP for processing (eg fusion) to obtain an image of the target object in a specific format. Further, optionally, the ISP can transmit the image of the target object to the display screen for display.

Embodiment 2

As shown in FIG. 10 , it is a schematic structural diagram of another camera module provided by the present application. The camera module may include a second optical lens assembly 1001, a third optical lens assembly 1002, a first image sensing assembly 1003 and a second image sensing assembly 1004, and the first image sensing assembly 1003 and the second image sensing assembly The photosensitive area of 1004 is the same, and the resolution of the first image sensing assembly 1003 is greater than that of the second image sensing assembly 1004; the second optical lens assembly 1001 is used to receive the light from the target object, and to the first image sensing The component 1003 transmits the third light; the third optical lens component 1002 is used to receive the light from the target object and transmit the fourth light to the second image sensing component 1004 .

Based on the above camera module, through the second optical lens assembly and the third optical lens assembly, the camera module can realize dual-channel imaging, and the second optical lens assembly can transmit the third light from the target object to the first image sensor component, the third optical lens component can transmit the fourth light from the target object to the second image sensing component; since the sensitivity and resolution of the image sensing component are fixed when the photosensitive area is fixed, therefore, the first The sensitivity of one image sensing element is less than the sensitivity of the second image sensing element. That is to say, the first image sensing component has high resolution and low sensitivity to light, is not easily overexposed to bright light, can identify high-brightness light, and can be used for imaging of bright targets; The sensor component has lower resolution and higher sensitivity to light, can recognize low-brightness light, and can be used for imaging of dim targets. In this way, the camera module can image both dim targets and bright targets, thereby helping to improve the dynamic range of the camera module.

Here, the information carried by the third light ray is the same as the information carried by the light ray from the target object, and the information carried by the fourth light ray is also the same as the information carried by the light ray from the target object. Exemplarily, the third light ray is a part or all of the light ray from the target object, and the fourth light ray is also a part or all of the light ray from the target object.

It should be noted that in the second embodiment, the distance between the first optical lens assembly and the second optical lens assembly is relatively close, so the field of view of the second optical lens assembly is the same as the field of view of the third optical lens assembly, or the parallax can be ignored. In other words, the light from the target object received by the second optical lens assembly is the same as the light from the target object received by the third optical lens assembly, or the difference is negligible.

The respective functional components and structures shown in FIG. 10 are introduced and described below to give an exemplary specific implementation solution. For the convenience of description, the second optical lens assembly, the third optical lens assembly, the first image sensing assembly and the second image sensing assembly in the following description are not marked.

6. The second optical lens assembly and the third optical lens assembly

In a possible implementation manner, the second optical lens assembly can receive the light from the target object, and by changing the propagation direction of the light from the target object, transmits the third light to the first image sensing assembly; the third optical lens The component can receive the light from the target object, and propagate the fourth light to the second image sensing component by changing the propagation direction of the light from the target object.

Further, optionally, both the second optical lens assembly and the third optical lens assembly may transmit glare (eg, glare reflected from the road surface) into the camera module.

In a possible implementation manner, the f-number (i.e., the F-number) of the second optical lens assembly is greater than the f-number of the third optical lens assembly. It can also be understood that the intensity of the third light ray transmitted from the target object through the second optical lens assembly is smaller than the intensity of the fourth light ray transmitted through the third optical lens assembly. In other words, the intensity of the third light ray is smaller than that of the fourth light ray. It should be understood that a small aperture number and a large amount of light transmission are conducive to imaging of dim and weak light; a large number of apertures and a small amount of light transmission are conducive to imaging of bright light.

In a possible implementation manner, the ratio of the intensities of the third light and the fourth light may be changed through the aperture number of the second optical lens assembly and the aperture number of the third optical lens assembly. For example, the ratio of the intensities of the third light and the fourth light may be 2:8; for another example, the ratio of the intensities of the third light to the fourth light is 1:9; for another example, the intensities of the third light and the fourth light The ratio is 4:6. It can also be understood that the intensity of the third light received by the first image sensing assembly is smaller than the intensity of the fourth light received by the second image sensing assembly.

In a possible implementation manner, the second optical lens assembly and the third optical lens assembly may both be optical lens assemblies with a fixed aperture, wherein the second optical lens assembly is a small aperture optical lens assembly, and the third optical lens assembly is Large aperture optical lens assembly.

In another possible implementation manner, both the second optical lens assembly and the third optical lens assembly are optical lens assemblies with adjustable aperture, and the adjustable aperture number of the second optical lens assembly is greater than the aperture number of the third optical lens assembly .

Through the second optical lens assembly and the third optical lens assembly, the camera module can realize dual-channel imaging, and the amount of light passing through different channels can be controlled by the aperture of the corresponding optical lens assembly. The second optical lens assembly with a large aperture number cooperates with the high-resolution first image sensing assembly, and the third optical lens assembly with a small aperture number cooperates with the low-resolution second image sensing assembly, thereby helping to improve the camera module dynamic range.

Further, optionally, the second optical lens assembly and the third optical lens assembly have the same focal length. In this way, the third light passing through the second optical lens assembly and the fourth light passing through the third optical lens assembly can be converged on the same plane, so that the first image sensing assembly and the second image sensing assembly can be arranged in the same plane on the substrate (see Figure 11 below), which helps simplify the assembly of the camera module. Based on this, the camera module of the second embodiment can be applied to a fixed-focus imaging scene.

In a possible implementation manner, the image plane circles of the second optical lens assembly and the third optical lens assembly are the same, so that the effective areas used by the first image sensing assembly and the second image sensing assembly for imaging are consistent of.

It should be noted that, for the structures of the second optical lens assembly and the third optical lens assembly, reference may be made to the description of the aforementioned first optical lens assembly, which will not be repeated here. In addition, the structures of the second optical lens assembly and the third optical lens assembly may be the same or different, which are not limited in this application.

Further, optionally, in order to minimize the height of the camera module, the optical lenses in the second optical lens assembly and/or the third optical lens assembly may be cut in the height direction of the camera module.

7. The first image sensing assembly and the second image sensing assembly

For the introduction of the first image sensing assembly and the second image sensing assembly, reference may be made to the foregoing related descriptions, which will not be repeated here. Specifically, the first ray can be replaced by the third ray, the second ray can be replaced by the fourth ray, the information of the first image can be replaced by the information of the fourth image, the information of the second image can be replaced by the information of the fifth image, The information of the third image is replaced with the information of the sixth image, etc.

The camera module in the second embodiment may further include a second polarizer. Further, optionally, the camera module may further include a second processing component. The second polarizer and the second processing component are respectively introduced below.

Eight, the second polarizer

The second polarizer is located between the second optical lens assembly and the first image sensing assembly (see FIG. 11 below). The second polarizing plate is used for allowing the light parallel to the polarization direction of the second polarizing plate to pass through, and the third light passing through the second polarizing plate is converged on the first image sensing component. It can also be understood that the second polarizer allows the third light rays that are parallel to the polarization direction of the second polarizer to pass through, and the rest are filtered out. In this way, the intensity of the third light entering the first image sensing element can be further reduced, thereby further reducing the overexposure of the first image sensing element to bright light, thereby further improving the dynamic range of the camera module. It should be noted that the third light rays passing through the second polarizer are a part of the third rays entering the second polarizer (ie, the part of the third rays parallel to the polarization direction of the second polarizer).

Further, optionally, the polarization direction of the second polarizer is perpendicular to the main polarization direction of the glare transmitted through the second optical lens assembly, and the second polarizer is introduced into the optical path of the high-resolution first image sensing assembly, And the polarization direction of the second polarizer is designed to be perpendicular to the main polarization direction of the received glare, which can eliminate or reduce the glare entering the high-resolution first image sensing component (the specific principle can be combined with the polarization state of the glare and the aforementioned Figure 2). description), so that a glare-free and high dynamic range image can be obtained.

With reference to the above FIG. 7 , the resolution of the first image sensing component is 8M, the resolution of the second image sensing component is 2M as an example, and the ratio of the intensity of the first light to the second light is 2:8 as an example . Table 4 exemplarily shows the relationship between the dynamic range of the camera module based on the second embodiment and the dynamic range of the camera module based on the prior art.

Table 4 is based on the dynamic range relationship between the camera module of the second embodiment and the camera module based on the prior art

When the camera module further includes a second polarizer, in combination with the above Table 4, similar to the analysis in Table 1 above, the overexposure prevention capability of the first image sensing component for high-brightness targets is improved by 2 times, and the second image sensor The low-light capability of the component for dim targets is increased by 4 times, and the dynamic range of the camera module is 10log8.

Further, optionally, the relationship between the ratio of the intensities of the third light rays and the fourth light rays and the improvement effect of the dynamic range can be found in Table 3 above, which will not be repeated here.

Nine, the second processing component

In a possible implementation manner, the second processing component may receive information of the fourth image from the first image sensing component, and receive information of the fifth image from the second image sensing component, and according to the fourth image and the information of the fifth image to obtain the image of the target object. Specifically, the second processing component is configured to upsample the information of the fifth image to obtain the information of the sixth image, and the resolution of the sixth image corresponding to the information of the sixth image is the fourth image corresponding to the information of the fourth image The resolution is the same; the information of the fourth image and the information of the sixth image are fused to obtain the image of the target object.

Here, for a possible example of the second processing component, reference may be made to the introduction of the first processing component, which will not be repeated here.

It should be noted that the camera module of the second embodiment may further include an infrared (infrared radiation, IR) filter, and the IR filter may be located between the second optical lens assembly and the first image sensing assembly, and /or, between the third optical lens assembly and the second image sensing assembly. For the introduction of the IR filter, reference may be made to the foregoing related descriptions, which will not be repeated here.

Based on the above content, a specific implementation manner of the camera module in the second embodiment above is given below in conjunction with the specific hardware structure. In order to further understand the structure of the above-mentioned camera module and the formation process of the image of the target object.

As shown in FIG. 11 , it is a schematic structural diagram of another camera module provided by the present application. The camera module may include a second optical lens assembly, a third optical lens assembly, a first image sensing assembly, a second image sensing assembly, a second optical lens assembly between the second optical lens assembly and the first image sensing assembly a polarizer and a second processing component. The first image sensing component is taken as an example in (a) in the above-mentioned FIG. 7 , and the second image sensing component is taken as an example in the above-mentioned (b) in FIG. 7 . In this example, the second optical lens assembly and the third optical lens assembly can be represented by a single lens, and for the specific structure, please refer to the foregoing description of the first optical lens assembly.

For possible implementations of the second optical lens assembly, the third optical lens assembly, the first image sensing assembly, the second image sensing assembly, the second polarizer, and the second processing assembly, please refer to the foregoing related descriptions respectively, here It will not be repeated.

Based on the camera module shown in Figure 11, the light from the target object passes through the second optical lens assembly to obtain the third light, and the third light is propagated to the first image sensing assembly; the light from the target object passes through the third optical lens assembly The fourth light is obtained, the fourth light is propagated to the second polarizer, and then propagated to the second image sensing component after passing through the second polarizer. The first image sensing component converts the third light into a fourth electrical signal (that is, the information of the fourth image), the second image sensing component converts the fourth light into a fifth electrical signal (the information of the fifth image), and the The four electrical signals can be converted into a third digital image signal after A/D, the fifth electrical signal can be converted into a fourth digital image signal after A/D, and both the third digital image signal and the fourth digital image signal are transmitted Go to the ISP for processing (eg fusion) to obtain an image of the target object in a specific format. Further, optionally, the ISP can transmit the image of the target object to the display screen for display.

Based on the structure and functional principle of the camera module described above, the present application can also provide a terminal device. The terminal device may include the camera module in the first embodiment above or may also include the camera module in the second embodiment above. Further, optionally, the terminal device may further include a memory and a processor, where the memory is used for storing programs or instructions; the processor is used for calling the programs or instructions to control the above-mentioned camera module to acquire the image of the target object. It can be understood that the terminal device may also include other devices, such as a wireless communication device, a touch screen, a display screen, and the like.

As shown in FIG. 12 , it is a schematic structural diagram of a terminal device provided by this application. The terminal device 1200 may include a processor 1201, a memory 1202, a camera module 1203, a display screen 1204, and the like. It should be understood that the hardware structure shown in FIG. 12 is only an example. The terminal device to which the present application applies may have more or fewer components than the terminal device shown in FIG. 12 , may combine two or more components, or may have different component configurations. The various components shown in Figure 12 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The processor 1201 may include one or more processing units. For example, the processor 1201 may include an application processor 1201 (application processor, AP), a graphics processor 1201 (graphics processing unit, GPU), an image signal processor 1201 (image signal processor, ISP), a controller, a digital signal processor 1201 (digital signal processor, DSP) and so on. Wherein, different processing units may be independent devices, or may be integrated in one or more processors 1201 .

The memory 1202 may be random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically EPROM, EEPROM), registers, hard disk, removable hard disk, CD-ROM or any other form of storage medium well known in the art. An exemplary memory 1202 is coupled to the processor 1201 such that the processor 1201 can read information from, and write information to, the memory 1202 . Of course, the memory 1202 may also be a component of the processor 1201 . Of course, the processor 1201 and the memory 1202 may also exist in the terminal device as discrete components.

The camera module 1203 can be used to capture moving and still images, and the like. In some embodiments, the terminal device may include one or N camera modules 1203, where N is a positive integer. For the introduction of the camera module 1203, reference may be made to the descriptions of the foregoing embodiments, which will not be repeated here.

When the camera module 1203 is applied as a vehicle camera module, based on the functions of the vehicle camera module, it can be divided into a driving assistance camera module, a parking assistance camera module and an in-vehicle driver monitoring camera module. Driving assistance camera modules are used for driving records, lane departure warning, door opening warning, blind spot monitoring and traffic sign recognition. Driving assistance camera modules include intelligent forward vision (such as monocular/binocular/trinocular), which can be used for dynamic object detection (vehicles, pedestrians), static object detection (traffic lights, traffic signs, lane lines, etc.) and passable Space division, etc.; side view assistance (such as wide-angle), used to monitor dynamic targets in the blind area of the rearview mirror during driving; night vision assistance (such as night vision camera), which can be used at night or in other situations with poor light. to achieve the detection of target objects. The parking assist camera module can be used for reversing image/360° surround view, 360° surround view (such as wide-angle/fisheye), mainly for low-speed close-up perception, and can form a seamless 360-degree panoramic top view around the vehicle. The in-vehicle driver monitoring camera module mainly provides one or more layers of early warning for dangerous situations such as driver fatigue, distraction, and irregular driving. Based on the different installation positions of the vehicle camera module in the terminal device, it can be divided into a front-view camera module, a side-view camera module, a rear-view camera module and a built-in camera module.

Display screen 1204 may be used to display images, video, and the like. Display screen 1204 may include a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode, or an active matrix organic light emitting diode (active-matrix organic light). emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the terminal device may include one or Q display screens 1204 , where Q is a positive integer greater than one. For example, the terminal device may implement a display function through a GPU, a display screen 1204, a processor 1201, and the like.

Exemplarily, the terminal device may be, for example, a vehicle, a smart phone, a smart home device, a smart manufacturing device, a robot, an unmanned aerial vehicle, or an intelligent transportation device (such as an AGV or an unmanned transportation vehicle, etc.).

Based on the above content and the same concept, the present application provides an imaging method, please refer to the introduction of FIG. 13 . The imaging method can be applied to any camera module shown in the first embodiment. It can also be understood that the imaging method can be implemented based on any camera module shown in the first embodiment above.

As shown in Figure 13, the imaging method includes the following steps:

Step 1301, receiving light from the target object.

This step 1301 can be implemented by the first optical lens assembly, which may refer to the above-mentioned introduction of the first optical lens assembly receiving light from the target object, and details are not repeated here.

Step 1302, splitting the light of the target object to obtain the first light and the second light, and propagating the first light to the first image sensing component and the second light to the second image sensing component.

Here, the ratio of the intensities of the first light ray to the second light ray is less than 1.

This step 1302 may be implemented by a light splitting component. For possible implementation manners, reference may be made to the foregoing related descriptions, which will not be repeated here.

In a possible implementation manner, photoelectric conversion is performed on the first light to obtain the information of the first image; photoelectric conversion is performed on the second light to obtain the information of the second image; according to the information of the first image and the information of the second image information to generate an image of the target object. Further, optionally, the information of the second image can be upsampled to obtain the information of the third image, and the information of the first image and the information of the third image can be fused to obtain the image of the target object, wherein the information of the third image is The corresponding third image has the same resolution as the first image corresponding to the information of the first image. This process may be implemented by the first processing component, and for possible implementation manners, reference may be made to the relevant description of the aforementioned first processing component, which will not be repeated here.

Based on the above content and the same concept, the present application provides an imaging method, please refer to the introduction of FIG. 14 . The imaging method can be applied to any camera module shown in the second embodiment above. It can also be understood that the imaging method can be implemented based on any camera module shown in the second embodiment above.

As shown in Figure 14, the imaging method includes the following steps:

Step 1401, receiving light from the target object.

Step 1402, propagating a third light ray to the first image sensing assembly, and propagating a fourth light ray to the second image sensing assembly.

Both steps 1401 and 1402 can be implemented by the second optical lens assembly and the third optical lens assembly. For possible implementations, please refer to the introduction of the second optical lens assembly and the third optical lens assembly, which will not be repeated here.

In a possible implementation manner, photoelectric conversion may be performed on the received third light to obtain information of the fourth image; photoelectric conversion may be performed on the received fourth light to obtain information of the fifth image; The information of the image and the information of the fifth image are used to obtain the image of the target object. Specifically, the information of the fifth image is up-sampled to obtain the information of the sixth image, and the information of the fourth image and the information of the sixth image are fused to obtain the image of the target object, wherein the information of the sixth image corresponds to the sixth image. The resolution of the image is the same as the resolution of the fourth image corresponding to the information of the fourth image. This process may be implemented by the second processing component, and for possible implementation manners, reference may be made to the relevant description of the foregoing second processing component, which will not be repeated here.

It should be noted that the above imaging method can be applied to the Internet of Vehicles, such as vehicle-to-everything (V2X), long term evolution-vehicle (LTE-V), vehicle-to-everything (vehicle to everything) , V2V) and so on.

Some steps in the methods in the embodiments of the present application may be implemented by means of hardware, and may also be implemented by means of a processor executing software instructions. Software instructions can be composed of corresponding software modules, and software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (programmable ROM) , PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically EPROM, EEPROM), registers, hard disks, removable hard disks, CD-ROMs or known in the art in any other form of storage medium. An exemplary storage medium is coupled to the processor, such that the processor can read information from, and write information to, the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium may reside in an ASIC. Alternatively, the ASIC may be located in a network device or in an end device. Of course, the processor and storage medium may also exist as discrete components.

In the various embodiments of the present application, if there is no special description or logical conflict, the terms and/or descriptions between different embodiments are consistent and can be referred to each other, and the technical features in different embodiments are based on their inherent Logical relationships can be combined to form new embodiments.

In this application, "vertical" does not mean absolute vertical, and a certain error may be allowed. "And/or", which describes the association relationship of the associated objects, indicates that there can be three kinds of relationships, for example, A and/or B, which can indicate: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A, B can be singular or plural. In the text description of this application, the character "/" generally indicates that the contextual objects are in an "or" relationship. In the formula of this application, the character "/" indicates that the related objects before and after are a "division" relationship. Also, in this application, the word "exemplarily" is used to mean serving as an example, illustration, or illustration. Any embodiment or design described in this application as "exemplary" should not be construed as preferred or advantageous over other embodiments or designs. Alternatively, it can be understood that the use of the word example is intended to present concepts in a specific manner, and not to limit the application.

It can be understood that, various numbers and numbers involved in the present application are only for the convenience of description, and are not used to limit the scope of the embodiments of the present application. The size of the sequence numbers of the above processes does not imply the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic. The terms "first", "second" and similar expressions are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, eg, comprising a series of steps or elements. A method, system, product or device is not necessarily limited to those steps or units expressly listed, but may include other steps or units not expressly listed or inherent to the process, method, product or device.

Although the application has been described in conjunction with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made therein without departing from the spirit and scope of the application. Accordingly, the present specification and drawings are merely illustrative of the approaches defined by the appended claims, and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

A camera module is characterized by comprising a first optical lens assembly, a light splitting assembly, a first image sensing assembly and a second image sensing assembly, the first image sensing assembly and the second image sensing assembly The photosensitive areas of the components are the same, and the resolution of the first image sensing component is greater than the resolution of the second image sensing component;

the first optical lens assembly for receiving light from the target object;

The spectroscopic component is used for splitting the light propagating through the first optical lens component to obtain a first light ray and a second light ray, and propagating the first light ray to the first image sensing component, and to the The second image sensing assembly propagates the second light.
The camera module of claim 1, wherein the camera module further comprises a first polarizer located between the light splitting component and the first image sensing component.
The camera module of claim 2, wherein the polarization direction of the first polarizer is perpendicular to the main polarization direction of the glare propagated through the first optical lens assembly.
The camera module according to any one of claims 1 to 3, wherein a ratio of the intensity of the first light to the second light is less than 1.
The camera module according to any one of claims 1 to 4, wherein the first image sensing component is used to perform photoelectric conversion on the received first light to obtain information of the first image ;

The second image sensing component is used to perform photoelectric conversion on the received second light to obtain information of the second image;

Wherein, the information of the first image and the information of the second image are used to form the image of the target object.
The camera module according to any one of claims 1 to 5, wherein the camera module further comprises a first processing component for:

receiving information from the first image of the first image sensing assembly, and receiving information from the second image of the second image sensing assembly;

An image of the target object is generated according to the information of the first image and the information of the second image.
The camera module of claim 6, wherein the first processing component is used for:

Up-sampling the information of the second image to obtain information of the third image, the resolution of the third image corresponding to the information of the third image and the resolution of the first image corresponding to the information of the first image same;

The image of the target object is obtained by fusing the information of the first image and the information of the third image.
A terminal device, comprising the camera module according to any one of claims 1 to 7.
The terminal device according to claim 8, wherein the terminal device comprises any one of the following:

Smartphones, vehicles, smart home equipment, smart manufacturing equipment, robots, drones, mapping equipment or smart transportation equipment.
An imaging method, characterized in that it is applied to a camera module, the camera module includes a first image sensing component and a second image sensing component, the first image sensing component and the second image sensor component The photosensitive areas of the sensing components are the same, and the resolution of the first image sensing component is greater than the resolution of the second image sensing component;

Receive light from the target object;

Splitting the light of the target object to obtain first light and second light, and propagating the first light to the first image sensing assembly, and propagating the second light to the second image sensing assembly light.
The method of claim 10, wherein the camera module further comprises a first polarizer located between the light splitting component and the first image sensing component.
The method of claim 11, wherein the polarization direction of the first polarizer is perpendicular to the main polarization direction of the glare propagating through the first optical lens assembly.
The method according to any one of claims 10 to 12, wherein the ratio of the intensity of the first light ray to the second light ray is less than 1.
The method according to any one of claims 10 to 13, wherein the method further comprises:

performing photoelectric conversion on the first light to obtain information of the first image;

performing photoelectric conversion on the second light to obtain information of the second image;

An image of the target object is generated according to the information of the first image and the information of the second image.
The method of claim 14, wherein the generating the image of the target object according to the information of the first image and the information of the second image comprises:

Up-sampling the information of the second image to obtain information of the third image, the resolution of the third image corresponding to the information of the third image and the resolution of the first image corresponding to the information of the first image same;

The image of the target object is obtained by fusing the information of the first image and the information of the third image.