JP2024131783A - Photodetector, distance measuring method, and distance measuring system - Google Patents

Abstract

[Problem] The present disclosure provides a light detection element, a measurement method, and a distance measurement system that enable measurements to be made in a shorter time with less power consumption. [Solution] According to the present disclosure, a light detection element is provided that includes a light receiving unit having a plurality of light receiving elements that receive irradiated light reflected by an object, and a control circuit that transmits an information signal including information for controlling the light emission method in frames N and after based on the reaction state of the plurality of light receiving elements in the Nth frame to an irradiation control unit that controls a light emitting unit that emits irradiated light. [Selected Figure] Figure 3

Description

This disclosure relates to a light detection element, a ranging method, and a ranging system.

Range measurement systems that measure the distance to an object (target object) based on ToF (Time of Flight) are commonly known. TOF generally includes direct TOF (dTOF) and indirect TOF (iTOF). Direct ToF emits multiple pulsed lights from a laser array, and detects photons reflected from the object on which each pulsed light is irradiated using a light receiving element called a SPAD (Single Photon Avalanche Diode).

This technology converts the carriers generated by this process into an electrical signal pulse using avalanche multiplication, and inputs this into a TDC (Time to Digital Converter) to measure the arrival time of the reflected light and calculate the distance to the object. On the other hand, indirect ToF measures the time of flight of light by emitting pulsed light from a light receiving element, detecting the charge generated by the light reflected from the object, and utilizing a semiconductor element structure in which the amount of charge stored changes depending on the timing of the light arrival. Attempts are being made to incorporate a ranging system using this SPAD into a surveillance system.

JP 2021-120630 A

However, a two-dimensional observation image is required to extract the region of interest (ROI) within the distance measurement range. This means that the host computer is constantly in operation, which can lead to high power consumption and a long time required for the fusion process between the two-dimensional observation image and the distance image using SPAD.

Therefore, this disclosure provides a light detection element, a distance measurement method, and a distance measurement system that enable measurements to be performed in a shorter time with less power consumption.

In order to solve the above problems, according to the present disclosure,
a light receiving section having a plurality of light receiving elements that receive light reflected by an object;
a control circuit that transmits an information signal including information for controlling a light emission method in frames N and thereafter based on a reaction state of the plurality of light receiving elements in the Nth frame to an irradiation control unit that controls a light emitting unit that emits the irradiation light;
A light detecting element is provided, comprising:

The light emission method may limit the number of emitted lights.

The control circuit may limit the number of lights emitted for each area of the light-emitting unit.

The light emission method may include stopping the light emission for a predetermined period of time.

The control circuit may stop the light emission for each area of the light-emitting section.

The irradiation light is irradiated a predetermined number of times,
When the number of times that the plurality of light receiving elements receive light exceeds a predetermined threshold, the control circuit may output a recovery signal to a main control unit.

The number of times the light is received may be set to a predetermined time range from the timing of irradiation of the light.

The predetermined time range may be set according to the distance measurement range.

The irradiated light may be laser light, and the reflected light may be photons.

The light emitting unit has a plurality of light emitting elements,
a light receiving surface of the light receiving unit is divided into a plurality of element group regions,
the plurality of light-emitting elements are associated with the plurality of element group regions based on their positions;
The number of emitted lights may be limited for each of the elements associated with the plurality of element group regions.

The control circuit may limit the number of emissions for each element associated with the plurality of element group regions based on the reaction state of each of the plurality of element group regions.

The light emitting unit has a plurality of light emitting elements,
a light receiving surface of the light receiving unit is divided into a plurality of element group regions,
the plurality of light-emitting elements are associated with the plurality of element group regions based on their positions;
The light emission may be stopped for each of the elements associated with the plurality of element group regions.

The control circuit may stop light emission for each of the elements associated with the plurality of element group regions based on the reaction state of each of the plurality of element group regions.

The irradiation light is irradiated a predetermined number of times,
When the number of times that the plurality of light receiving elements receive light exceeds a predetermined threshold, the control circuit may output a recovery signal to a main control unit.

The irradiation light is irradiated a predetermined number of times,
adding up the number of times that the light is received by each of the plurality of light receiving elements in accordance with the elapsed time from the irradiation timing of the irradiation light;
When the added number exceeds a predetermined threshold value,
The control circuit may limit the number of emitted lights.

The irradiation light is irradiated a predetermined number of times,
adding up the number of times that the light is received by each of the plurality of light receiving elements in accordance with the elapsed time from the irradiation timing of the irradiation light;
When the added number exceeds a predetermined threshold value,
The control circuit may stop the light emission for a predetermined period.

The frame may correspond to a predetermined control period.

In order to solve the above problems, according to the present disclosure, there is provided a method for detecting a reflected light from an object using a plurality of light receiving elements;
a control step of transmitting an information signal including information for controlling a light emission method in frames N and thereafter based on a reaction state of the plurality of light receiving elements in the Nth frame to an irradiation control unit that controls a light emitting unit that emits the irradiation light;
A method for measuring distance is provided, comprising:

In order to solve the above problems, according to the present disclosure, there is provided a light emitting unit having a light emitting unit that emits irradiation light,
an illumination control unit that controls the light emitting unit;
A light detection unit;
A ranging system comprising:
The light detection unit includes:
a light receiving section having a plurality of light receiving elements that receive light reflected by an object;
a control circuit that transmits an information signal including information for controlling a light emission method in frames N and thereafter based on a reaction state of the plurality of light receiving elements in the Nth frame to an irradiation control unit that controls a light emitting unit that emits the irradiation light;
A ranging system is provided having the following:

A main control unit capable of controlling the light detection unit,
The main control unit may be in a power saving state when the light receiving unit controls the light emission method.

1 is a block diagram showing an example of the configuration of a distance measuring system according to an embodiment of the present technology. FIG. 2 is a diagram showing a configuration example of a light-emitting unit. FIG. 2 is a block diagram illustrating details of a light emission control device and a light detection unit. FIG. FIG. FIG. 13 is a diagram showing the relationship between a laser trigger signal and a histogram. FIG. 13 is a diagram showing an example in which an object is present in an area corresponding to bank 0; FIG. 4 is a diagram for explaining a first mode. FIG. 11 is a diagram showing a more detailed example of a control sequence in the first mode. FIG. 4 is a diagram for explaining a second mode. FIG. 11 is a diagram showing a more detailed example of a control sequence in the second mode. FIG. 4 is a diagram showing an example of a register configuration stored in a register; 4 is a diagram showing an example of a data configuration of an information signal generated by a stop information generating circuit; 13A and 13B are diagrams showing example histograms generated for each element group in each frame. FIG. 13 is a diagram showing an example of a histogram generated for each element group in each frame, where there is a peak. FIG. 13 is a diagram showing an example in which determination information is recorded in a register. FIG. 13 is a diagram showing an example of non-return determination information recorded in a register. 11 is a diagram showing an example of a sequence including an AP return process; 4 is a flowchart showing a processing example according to the embodiment. FIG. 13 is a diagram showing a peak detection range in the case of a short distance. FIG. 13 is a diagram showing a peak detection range in the case of a long distance. FIG. 13 is a diagram showing an example of intensity change in the case of a long distance; FIG. 4 is a diagram showing a determination process of a processing circuit. FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle control system. FIG. 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit.

Below, embodiments of the light detection element, the distance measurement method, and the distance measurement system will be described with reference to the drawings. The following description will focus on the main components of the light detection element, the distance measurement method, and the distance measurement system, but the light detection device and the distance measurement system may have components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

First Embodiment
Fig. 1 is a block diagram showing an example of the configuration of a distance measuring system according to an embodiment of the present technology. As shown in Fig. 1, the distance measuring system 1 according to the present embodiment is a system that emits pulsed light and can determine whether or not an object Obj (target object or subject) is present in a monitoring area based on the reflected light from the object Obj irradiated with the pulsed light. That is, the distance measuring system 1 includes, for example, a light emitting unit 100, a light emitting control unit (LDD) 200, a light detection unit 300, an imaging unit 400, and a main control unit (AP) 500.

Figure 2 is a diagram showing an example of the configuration of the light-emitting unit 100. As shown in Figure 2, the light-emitting unit 100 is, for example, a laser array unit, and has a plurality of light-emitting elements 110 arranged two-dimensionally along a light-emitting surface. The light-emitting unit 100 is capable of irradiating the object Obj with laser light as irradiation light. This irradiation light is generated by the emission of the plurality of light-emitting elements 110, and is irradiated in a predetermined direction.

Each light-emitting element 110 outputs laser light (also called irradiation light) of a predetermined wavelength at a predetermined pulse repetition period (also called PRI period) according to drive control by the light-emitting control unit 200. For each light-emitting element 110, for example, a vertical cavity surface-emitting laser (VCSEL) can be used. However, this is not limited to this, and various light sources capable of emitting light of a predetermined wavelength can be used.

Furthermore, among the plurality of light-emitting elements 110, a plurality of light-emitting elements 110m correspond to each position of element groups MP0 to MP13 (see FIG. 4) described below. These plurality of light-emitting elements 110m can selectively emit light in accordance with each position of MP0 to MP13 (see FIG. 4).

As shown in FIG. 1 again, the light emission control unit (LDD) 200 is, for example, a laser diode driver (LDD) and controls the area of the light emitting element of the light emitting unit 100 and the light emission timing according to the control of the light detection unit 300 or the main control unit 500. That is, the light emission control unit 200 drives and controls each light emitting element 110 of the light emitting unit 100. The light emission control unit 200 may be configured to be capable of independently driving each light emitting element 110, or may be configured to be capable of independently driving the light emitting element 110 for each area. Alternatively, the light emitting element 110 can be independently driven for each row or column. The details of the light emission control unit 200 will be described later. The combination of the light emitting unit 100 and the light emission control unit 200 may be referred to as a light source.

The light detection unit 300 is, for example, a direct TOF distance measurement unit. This light detection unit 300 can detect the region of interest (ROI: Region of Interest) of an object Obj (target or subject) without generating a distance image, based on an electrical signal pulse obtained by receiving reflected light with a light receiving element called a SPAD (Single Photon Avalanche Diode). The light detection unit 300 also outputs information for controlling the light emission method to the light emission control unit 200 based on the reaction state of the light receiving element. Furthermore, the light detection unit 300 can measure the distance to the object Obj and generate a distance image.

This embodiment will be described using a direct TOF distance measurement unit as an example, but is not limited to this. For example, it can be similarly applied to an indirect TOF distance measurement unit. The light emission control unit 200 and the light detection unit 300 can be configured integrally as a system on chip (SoC) such as a CMOS LSI, but is not limited to this. For example, the light emission control unit 200 and the light detection unit 300 may be configured as an LSI with some components separate from each other. Note that the light detection unit 300 in this embodiment corresponds to a light detection element. Details of the light detection unit 300 will also be described later.

The imaging unit 400 is, for example, a camera, and is capable of capturing a two-dimensional observation image using visible light or infrared light. The imaging unit 400 outputs the two-dimensional observation image to the main control unit 500.

The main control unit 500 is, for example, an application processor (AP), and controls the entire distance measurement system 1. For example, the main control unit 500 controls the light emission control unit 200, the light detection unit 300, and the image capture unit 400. When the light emission unit 100, the light emission control unit 200, and the light detection unit 300 are in monitoring mode, the image capture unit 400 and the main control unit 500 can transition to a power saving mode in which power consumption is limited. In addition, when the light detection unit 300 detects an object Obj, the image capture unit 400 and the main control unit 500 can transition from the power saving mode to a normal mode. Furthermore, the main control unit 500 can perform fusion processing of the two-dimensional image captured by the image capture unit 400 and the region (ROI) of the object Obj generated by the light detection unit 300, or fusion processing of the two-dimensional image and the distance image. The main control unit 500 may be referred to as a host computer.

Here, the light emission control unit 200 and the light detection unit 300 will be described in detail with reference to FIG. 3. FIG. 3 is a block diagram for explaining the light emission control unit 200 and the light detection unit 300 in detail. As shown in FIG. 3, the light emission control unit 200 controls the light emission method of each light emitting element 110m of the light emission unit 100 based on the information signal SPI supplied from the light detection unit 300. That is, the light emission control unit 200 has a control circuit 210, a drive circuit 220, and an external interface (IF) 230. Furthermore, the control circuit 210 has a register 210a and a laser emission threshold control circuit 210b.

The light detection unit 300 has a light receiving unit 310, a TDC 320, a processing circuit 330, a control circuit 340, a laser trigger generation circuit 350, a stop information generation circuit 360, and an external interface (IF) 370. Furthermore, the light receiving unit 310 has an element array unit 310a and a detection circuit 310b. The processing circuit 330 has a histogram generation circuit 330a, a determination circuit 330b, and a distance calculation circuit 330c, and the control circuit 340 has a register 340a. The control unit U100 is a unit that is mainly used for control when controlling the light emission method of each light emitting element 110m.

As described above, the distance measurement system 1 according to this embodiment has a monitoring mode and a normal mode. As described above, in the monitoring mode, the imaging unit 400 and the main control unit 500 can transition to a power saving mode.

As described above, in the monitoring mode, the light emitting unit 100, the light emitting control unit 200, and the light detecting unit 300 are in a monitoring state independent of the main control unit 500. In other words, the distance measuring system 1 according to this embodiment can determine whether or not an object Obj is present in the monitoring area based on information from the light detecting unit 300, without using a two-dimensional image captured by the imaging unit 400. In this case, it is also unnecessary to generate a two-dimensional distance image. This enables high-speed, power-saving observation. Furthermore, the monitoring mode according to this embodiment has two modes, according to the control of the light detecting unit 300.

The first mode is a frame thinning mode. The first mode is a mode in which the light emitting element 110m of the light emitting unit 100 corresponding to an area where there is no reflected light from the object Obj and the light receiving element of the light detection unit 300 are stopped, and the light emitting element 110m and/or the light receiving element are periodically restored to check whether there is reflected light from the object Obj. That is, in the first mode, the light emitting element 110m of the light emitting unit 100 corresponding to an area where there is no reflected light from the object Obj is stopped from emitting light for a certain period of time. In this way, the light emitting method is controlled so that the light emitting element 110m of the light emitting unit 100 corresponding to an area where there is no reflected light from the object Obj is stopped from emitting light for a certain period of time.

The second mode is a number-of-emissions limiting mode. The second mode is a mode in which the number of emissions of the light-emitting elements 110m of the light-emitting unit 100 corresponding to areas where there is no reflected light from the object Obj is reduced. In the second mode, the light-receiving elements of the light-detecting unit 300 maintain their driven state. This results in a faster response speed of judgment in the light-detecting unit 300 than in the first mode. That is, in the second mode, the number of emissions of the light-emitting elements 110m of the light-emitting unit 100 corresponding to areas where there is no reflected light from the object Obj is limited for a certain period of time. In this way, the light-emitting method is controlled so as to limit the number of emissions of the light-emitting elements 110m of the light-emitting unit 100 corresponding to areas where there is no reflected light from the object Obj for a certain period of time.

As shown in FIG. 3, the control circuit 210 controls the drive circuit 120, for example, according to control from the light detection unit 300 or the main control unit 500. In the monitoring mode, the light detection unit 300 controls the light emission unit 100 via the control circuit 210 based on the reaction state of the light receiving element in the light detection unit 300.

Register 210a stores control parameters for each control mode based on the user's settings. For example, register 210a stores the threshold value for the number of laser shots in the second mode. Laser shot threshold control circuit 210b controls the number of shots per unit time of light-emitting unit 100 via drive circuit 220 according to the information in register 210a.

The drive circuit 220 controls the light emission of the light emitting elements 110 of the light emitting unit 100 according to the control of the control circuit 210. This drive circuit 120 may include, for example, a plurality of drive circuits provided one-to-one with each of the light emitting elements 110. In this case, the light emitting unit 100 is configured such that, for example, each light emitting element 110 corresponds to each of the drive circuits.

Figure 4 is a schematic plan view of the element array section 310a of the light receiving section 310. The element array section 310a has a plurality of light receiving elements 311 arranged in a two-dimensional array along the light receiving surface. Furthermore, the element array section 310a has a detection circuit 310b. This light receiving element 311 is configured to include a SPAD.

The detection circuit 310b includes, for example, a readout circuit that corresponds one-to-one with each light receiving element 311, and reads out and outputs the detection of photons by each light receiving element 311. That is, the light receiving element 311 according to this embodiment is composed of a SPAD element in the light receiving element 311 and a readout circuit of the detection circuit 310b connected thereto. The light receiving element 311 may also be referred to as a pixel.

In FIG. 4, only the light receiving elements 311 selected from the multiple light receiving elements 311 as element groups MP0 to MP13 are shown in schematic form. In this way, the light receiving surface of the light receiving unit 310 is divided into the regions of element groups MP0 to MP13.

Each of the element groups MP0 to MP13 is, for example, a CMOS image sensor including multiple light receiving elements 311. That is, the element array section 310a is a CMOS image sensor including multiple SPADs arranged in a two-dimensional array. Each light receiving element 311 detects incoming light (photons) and converts the carriers generated by this into an electrical signal pulse using avalanche multiplication. A group of several adjacent SPADs (SPAD group) may be referred to as an element. For example, the element groups MP0 to MP13 in this embodiment are composed of a group of adjacent 2 x 3 elements. In the monitoring mode, a histogram can be generated for each of the multiple element groups MP0 to MP13.

When generating a normal distance image, a distance value can be generated for each light receiving element 311. Note that the multiple element groups MP0 to MP13 are configured as a collection of adjacent 2x3 elements, but are not limited to this. For example, the multiple element groups MP0 to MP13 may be configured as a collection of adjacent light receiving elements 311 in any number of arrangements (array patterns), such as 2x2, 3x2, 3x3, 6x6, or 9x9. The number of the multiple element groups MP0 to MP13 is also not limited to 14.

As described above, the detection circuit 310b includes, for example, a readout circuit that corresponds one-to-one with each light receiving element 311, and reads out and outputs the detection of photons by each light receiving element 311. Each readout circuit includes, for example, a function for detecting when each light receiving element 311 detects a photon and enters a quench state, and a function for quickly recharging each light receiving element 311 to its original state. In addition, it outputs a pulse (detection signal PXOUT) whose rising edge occurs when each light receiving element 311 detects a photon.

The TDC 320 measures the time of flight of photons from when the light emitting unit 100 emits light until the reflected light is detected by the SPAD element, and outputs the measured time as a digital output value TDCOUT. In other words, the time difference between the generation timing of the detection signal PXOUT output by the light receiving element 311 and the generation timing of a laser trigger signal (described later) is output as the output value TDCOUT.

In this way, the TDC320 has a TDC circuit for each element, and measures the elapsed time from the generation timing of a laser trigger signal (described later) or the irradiation timing of the irradiated light to the detection timing of the reflected light for each element with high resolution (for example, about 100 ps (picoseconds) period), and outputs the time obtained thereby as a digital output value TDCOUT. Note that the element in this embodiment may be each element when multiple element groups MP0 to MP13 are each treated as one element, or each element when one SPAD element is treated as one element.

In the monitoring mode, the processing circuit 330 determines whether or not an object Obj exists in each area corresponding to each element group MP0 to MP13 based on, for example, the output value TDCOUT of each element group MP0 to MP13 output from the TDC 320. When generating a normal distance image, the processing circuit 330 also calculates distance information to the subject for each light receiving element 311 output from the TDC 320.

More specifically, in the monitoring mode, the histogram generation circuit 330a generates a histogram for each of the element groups MP0 to MP13 based on the output value TDCOUT of each of the element groups MP0 to MP13 output from the TDC 320. For example, the histogram generation circuit 330a generates a histogram for each of the element groups MP0 to MP13 by adding the output value TDCOUT output from the TDC 320 for each of the element groups MP0 to MP13 to a count value (accumulated value) stored in the bin (BIN) corresponding to the sampling period.

In addition, in the normal imaging mode, the histogram generation circuit 330a generates a histogram for each light receiving element 311 based on the output value TDCOUT of each light receiving element 311 output from the TDC 320. For example, the histogram generation circuit 330a generates a histogram for each light receiving element 311 by adding the output value TDCOUT output from the TDC 320 for each light receiving element 311 to a count value (accumulated value) stored in a bin (BIN) corresponding to the sampling period. In other words, the histogram generation circuit 330a adds up the number of times a given light receiving element 110 receives light according to the elapsed time from the irradiation timing of the irradiated light.

The determination circuit 330b determines whether or not an object Obj exists in the corresponding region for each of the element groups MP0 to MP13 based on the histograms for each of the element groups MP0 to MP13 generated by the histogram generation circuit 330a. For example, the determination circuit 330b determines whether or not an object Obj exists in the corresponding region for each of the element groups MP0 to MP13 based on the shape of the histograms for each of the element groups MP0 to MP13 generated by the histogram generation circuit 330a.

More specifically, when a predetermined peak is present in the histogram for each of the element groups MP0 to MP13 generated by the histogram generation circuit 330a, i.e., when a predetermined peak is present in the number of times photons are detected, the determination circuit 330b determines that an object Obj is present in the corresponding region for each of the element groups MP0 to MP13. In this way, the histogram generation circuit 330a adds up the number of times a predetermined light receiving element 110 receives light according to the elapsed time from the timing of irradiation with the irradiated light, and the determination circuit 330b determines that an object Obj is present when the sum exceeds a predetermined threshold value.

In addition, in monitoring mode, the histogram generation circuit 330a can also generate histograms for each of the multiple element groups MP0 to MP13, which are grouped into banks 0 to 3 (described below). In this case, the judgment circuit 330b judges whether or not an object Obj exists in the corresponding area for each of banks 0 to 3, based on the shape of the histograms for banks 0 to 3 generated by the histogram generation circuit 330a.

The distance calculation circuit 330c calculates distance information for each light receiving element 311 based on the histogram generated by the histogram generation circuit 330a. For example, the distance calculation circuit identifies the peak of the cumulative value of each histogram (i.e., the peak of the number of photon detections), and generates distance information for each element based on the bin number of the identified peak. Note that, if multiple bin numbers are identified as the peak of the cumulative value, the distance information for the element may be calculated based on the center of gravity (average) obtained from the identified bin and its cumulative value. The distance calculation circuit 330c can also calculate distance information for each of the element groups MP0 to MP13 based on the histogram generated by the histogram generation circuit 330a. Similarly, the distance calculation circuit 330c can also calculate distance information for each of the banks 0 to 3 based on the histogram generated by the histogram generation circuit 330a.

In monitoring mode, the control circuit 340 can be controlled independently of the main control unit 500. That is, in monitoring mode, the control circuit 340 controls each component of the light detection unit 300. Based on the determination information of the determination circuit 330b, the control circuit 340 can control the drive of each of the element groups MP0 to MP13, the drive of each of the light emitting element groups 110m corresponding to the element groups MP0 to MP13, the TDC 320, and the processing circuit 330.

In addition, in the monitoring mode, the control circuit 340 can return the main control unit 500 from the power saving mode to the normal mode. That is, the control circuit 340 sends a return signal to the main control unit 50 when the reaction state of the element groups MP0 to MP13 to the reflected light reaches a predetermined state.

In addition, in normal mode, the control circuit 340 can generate one frame of a distance measurement image from the distance information of each light receiving element 311 generated by the distance calculation circuit 330c, and output the generated distance image to the main control unit 500 via the external I/F 370. The register 340a stores each control parameter, including the identification information of banks 0 to 3 indicating the combination of element groups MP0 to MP13 (see Figures 5 and 12).

The laser trigger generating circuit 350 generates a laser trigger signal that causes the light emitting unit 100 to output irradiation light, for example, according to control from the control circuit 340. As described above, the stop information generating circuit 360 outputs an information signal SPI to the control circuit 210 via the external I/Fs 370 and 230, for example, according to control from the control circuit 340, which stops the light emission of the light emitting element 110m corresponding to banks 0 to 3 described later, or limits the number of light emission. In addition, the stop information generating circuit 360 outputs an information signal SPI to the control circuit 210 via the external I/Fs 370 and 230, for example, according to control from the control circuit 340, which returns the light emission of the light emitting element 110m corresponding to banks 0 to 3 described later to a normal state. Communication between the external I/Fs 370 and 230 is possible using, for example, I2C (Inter-Integrated Circuit) and SPI (Stateful Packet Inspection).

[About banks]
5 is a diagram showing an example of banks. To each light-emitting element 110m (see FIG. 2), banks 0 to 3 are assigned according to the position on the light-emitting surface. The horizontal axis indicates time. From the top, there are bank information, a diagram showing the position of the light-emitting element 110m of the light-emitting unit 100, and a diagram showing the position of the light-receiving element 311 of the light-receiving unit 310. Note that the number of banks 0 to 3 according to this embodiment is 4, but is not limited to this. For example, the number of banks may be 8, 12, 24, etc.

In this embodiment, a bank is, for example, identification information for identifying which of the multiple light-emitting elements 110m and multiple light-receiving elements 311 are to be driven. For example, bank 0 indicates the identification information for MP0, MP2, MP4, and MP5 of the element group MP0 to MP13 (see FIG. 4 and FIG. 14 described later). Bank 0 also indicates the identification information for the light-emitting element 110m (see FIG. 2) that emits light in correspondence with these MP0, MP2, MP4, and MP5.

Similarly, for example, bank 1 indicates the identification information of MP1, MP3, and MP6 from among the element groups MP0 to MP13 (see FIG. 4). Bank 1 also indicates the identification information of the light-emitting element 110m that emits light in correspondence with these MP1, MP3, and MP6. Similarly, for example, bank 2 indicates the identification information of MP8, MP9, and MP12 from among the element groups MP0 to MP13 (see FIG. 4). Bank 2 also indicates the information of the light-emitting element 110m that emits light in correspondence with these MP8, MP9, and MP12. Similarly, for example, bank 3 indicates the identification information of MP10, MP11, and MP13 from among the element groups MP0 to MP13 (see FIG. 4). Bank 3 also indicates the identification information of the light-emitting element 110m that emits light in correspondence with these MP10, MP11, and MP13. The identification information of these banks 0 to 3 is stored in the register 340a.

As shown in FIG. 5, in this embodiment, the light-emitting elements 110m and light-receiving elements 311 corresponding to banks 0 to 3 are controlled in a time series. That is, when the light-emitting elements 110m and light-receiving elements corresponding to bank 0 are driven, the light-emitting elements 110m and light-receiving elements corresponding to banks 1 to 3 are stopped from being driven. This makes it possible to reduce power consumption. Also, in this embodiment, the time-series control sequence of banks 0 to 3 is referred to as one frame. For example, one frame means that the control sequence of banks 0 to 3 is repeated once.

[Relationship between laser trigger signal and histogram]
FIG. 6 is a diagram showing the relationship between a laser trigger signal and a histogram. FIG. 6(a) shows a laser trigger signal in bank 0 in monitoring mode. The horizontal axis shows time ta, and the vertical axis shows the laser trigger signal. When the laser trigger signal is at a high level, the light-emitting element 110m associated with bank 0 emits light. For example, the laser trigger generating circuit 350 outputs, for example, several thousand laser trigger signals during time t10 to the light-emitting element 110m associated with bank 0. That is, the period from when one high-level signal is output until the next high-level signal is output corresponds to the PRI period of the light-emitting unit 100. Note that FIG. 6 is an example in which neither stop information nor limiting information on the number of shots is included in the information signal SPI.

Figure 6 (b) shows an example histogram for bank 0 in monitoring mode. The horizontal axis indicates time ta, and the vertical axis corresponds to the occurrence frequency of the flight times of photons detected in element group MP0 measured by TDC320. The time axis ta and the time axis tb have different scales. For example, the time width of the histogram corresponds to the PRI period.

As described above, the histogram generation circuit 330a generates histograms for each of the element groups MP0, MP2, MP4, and MP5 based on the output values TDCOUT of each of the element groups MP0, MP2, MP4, and MP5 output from the TDC320. For example, the histogram generation circuit 330a generates each histogram for bank 0 by adding the output values TDCOUT output from the TDC320 for each of the element groups MP0, MP2, MP4, and MP5 to the count values (accumulated values) stored in the bins (BINs) corresponding to the sampling period. Similarly, the histogram generation circuit 330a generates histograms for each of the banks 1 to 3 by adding the output values TDCOUT output for each of the element groups corresponding to the banks 1 to 3 to the count values (accumulated values) stored in the bins (BINs) corresponding to the sampling period.

The time width of the bins of the histogram corresponds to, for example, the sampling period. The sampling period is a period for measuring the time (time of flight) from when the light-emitting unit 100 emits irradiation light to when the light-receiving unit 310 detects the incidence of reflected light. This sampling period is set to a period shorter than the PRI period of the light-emitting unit 100. For example, by shortening the sampling period, it is possible to estimate or calculate the time of flight of photons emitted from the light-emitting unit 100 and reflected by the subject with higher time resolution. This means that by increasing the sampling frequency, it is possible to estimate or calculate the distance to the subject with higher ranging resolution. In other words, if the time width of the histogram is the period from when one high-level signal of the laser trigger signal is output to when the next high-level signal is output, the time width of the bin corresponds to the number obtained by dividing the period from when one high-level signal is output to when the next high-level signal is output by the number of bins.

Figure 7 is a schematic diagram showing an example in which subject Obj10 is in the area corresponding to bank 0. For example, if subject Obj10 is in the area corresponding to bank 0, then as shown in Figure 6(b), a histogram peak P10 appears at the distance corresponding to subject Obj10. In other words, this peak P10 corresponds to the distance to subject Obj10.

For example, when performing distance calculation, if the time corresponding to peak P10 is t, then since the speed of light C is constant (C ≈ 300,000,000 m (meters)/s (seconds), the distance calculation circuit 330c estimates or calculates the distance L to the subject according to the following formula (1).
L=C×t/2 (1)

[First mode]
An example of a sequence in the first mode will be described with reference to FIG. 8 and FIG. 9. As described above, the first mode is a frame thinning mode. In the processing example of FIG. 8 and FIG. 9, the judgment circuit 330b performs a judgment process to judge the presence or absence of a histogram peak based on the histogram generated for each of the banks 0 to 3. As will be described later, the judgment circuit 330b can also perform judgment processes for each of the element groups MP0 to MP13 in parallel. For example, when performing judgment processing for bank 0, it is possible to perform judgment processing to judge the presence or absence of a histogram peak for each of the element groups MP0, MP2, MP4, and MP5 included in bank 0.

Figure 8 is a diagram for explaining the first mode. Figures 8(a) and 8(c) show a laser trigger signal in bank 0 in the first mode. The horizontal axis shows time ta, and the vertical axis shows the laser trigger signal. Figure 8(a) shows, for example, frame N=n, and Figure 8(c) shows, for example, frame N=n+1. Figures 8(b) and 8(d) show an example histogram in bank 0 in the first mode. The horizontal axis shows time ta, and the vertical axis corresponds to the occurrence frequency of the flight times of photons detected in element groups MP0, MP2, MP4, and MP5 measured by TDC320. Figure 8(b) shows, for example, frame N=n, and Figure 8(d) shows, for example, frame N=n+1.

As shown in FIG. 8(b), for example, the determination circuit 330b (see FIG. 2) determines that there is no subject in the area corresponding to bank 0 in frame N=n because peak P10 is not present. In this case, the control circuit 330 outputs an information signal SPI to the control circuit 210 via the stop information generation circuit 360, which stops the driving of the light-emitting element group corresponding to bank 0 in frame N=n+1. In this way, the control circuit 330 outputs an information signal SPI to the control circuit 210, which is information that controls the light emission method of the light-emitting unit 100 in frames after N=n, based on the reaction state of the element groups MP0 to MP13. The light emission method in this case is to stop light emission for a predetermined period of time.

This information signal SPI stops the emission of light from the light-emitting unit 100 in frame N=n+1, as shown in FIG. 8(c). In this case, as shown in FIG. 8(d), the generation of the histogram is also stopped in frame N=n+1. That is, when the number of times that the multiple light-receiving elements 311 corresponding to bank 0 receive light exceeds a predetermined threshold, the control circuit 330 stops the emission of the corresponding light-emitting element 110m of the light-emitting unit 100 for a predetermined period of time. The control circuit 330 also executes control to stop emission of light for a predetermined period for each area of the light-emitting unit 100 corresponding to banks 0 to 3.

Figure 9 is a diagram showing a more detailed control sequence example of the first mode. Here, an example is described in which, in frame N=n, the emission of the light-emitting element corresponding to the bank in which peak P10 does not exist is stopped at N=n+1, n+2. The horizontal axis is time, and from the top, the number N of sequences (Bank sequences) that repeat banks 0 to 3, the type of bank, the schematic positions of the light-emitting unit 100 and the light-receiving element and light-emitting element of the light-detecting unit 300, the output timing of the laser trigger signal output by the laser trigger generating circuit 350, the output timing of the information signal SPI output by the stop information generating circuit 360, the check state of the control circuit 330, the emission timing (laser) of the light-emitting element corresponding to each bank, the light-receiving state of the reflected light of the light-detecting unit 300 (SPAD reflected light state), the drive state (Enable) of the histogram generating circuit 330a, and the judgment state (peak detection) of the judgment circuit 330b are shown.

First, in bank sequence N=0, the light-emitting element groups defined by banks 0 to 3 emit light in chronological order in synchronization with the output timing of the laser trigger signal, in the order of banks 0 to 3. At the same time, the light-receiving element groups defined by banks 0 to 3 are also driven in chronological order.

The histogram generating circuit 330a generates histograms in time series corresponding to the light receiving element groups defined by banks 0 to 3. That is, the control circuit 330 outputs an enable signal to the histogram generating circuit 330a as an enable signal corresponding to the light receiving element groups defined by banks 0 to 3. Since all the histograms corresponding to banks 0 to 3 are valid, the determination circuit 330b determines whether or not a predetermined peak is present for each histogram, i.e., whether or not an object Obj is present. Since there is no reflected light from the object Obj in the light receiving element areas corresponding to banks 2 and 3, the predetermined peak is not generated in each histogram corresponding to banks 2 and 3. Therefore, the determination circuit 330b determines that there is no predetermined peak for each histogram corresponding to banks 2 and 3.

Then, the control circuit 330 outputs an information signal SPI, which includes information on the determination result of the determination circuit 330b, to the control circuit 210 via the stop information generation circuit 360 between frames N=n and N=n+1. This information signal SPI includes information to stop the emission of the light-emitting element groups corresponding to banks 2 and 3, since there is no specified peak in banks 2 and 3.

In this embodiment, a frame that outputs an information signal SPI is called a check frame, and a frame that does not output an information signal SPI is called a check skip (check skip). That is, in the first mode shown in FIG. 9, the information signal SPI is output once every three frames, and in the subsequent frames N=1 and 2, control is performed according to the information signal SPI output at N=0. The number of frames to be checked and skipped can be set according to the moving speed of the subject Obj to be monitored.

Next, in bank sequence N=1, since it is a check skip frame, the control circuit 210 executes control according to the information signal SPI in bank sequence N=0. That is, in the order of banks 0 to 1, the light-emitting element groups defined by banks 0 to 1 emit light in time series in synchronization with the output timing of the laser trigger signal. At this time, the light-receiving element groups defined by banks 0 to 1 are also driven in time series. On the other hand, based on the stop information included in the information signal SPI, the light-emitting element groups defined by banks 2 to 3 stop emitting light. At this time, the light-receiving element groups defined by banks 2 to 3 also stop driving. This makes it possible to stop driving the light-emitting element groups and light-receiving element groups corresponding to the area where the object Obj does not exist, making it possible to save power. On the other hand, since the light-emitting element groups and light-receiving element groups corresponding to the area where the object Obj was present continue to be driven, it is possible to determine whether the object Obj continues to exist in the area where the object Obj was present.

The histogram generating circuit 330a generates a histogram in time series corresponding to the light receiving element group defined by banks 0-1 under the control of the control circuit 330. That is, the control circuit 330 outputs an enable signal to the histogram generating circuit 330a as an enable signal corresponding to the light receiving element group defined by banks 0-1. On the other hand, the control circuit 330 outputs an disable signal to the histogram generating circuit 330a as an enable signal corresponding to the light receiving element group defined by banks 2-3.

Because the histograms corresponding to banks 0 to 1 are valid, the determination circuit 330b determines whether or not each histogram has a specified peak, i.e., whether or not subject Obj is present. Because there is no reflected light from subject Obj in the area of the light receiving element corresponding to banks 0 and 1, the histograms corresponding to banks 0 and 1 do not produce a specified peak. For this reason, the determination circuit 330b determines that the histograms corresponding to banks 0 and 1 do not have a specified peak.

On the other hand, since the histograms corresponding to banks 2 and 3 are invalid, the judgment circuit 330b does not make a judgment on each histogram. When the judgment circuit 330b does not make a judgment, it may output a default output indicating that a specified peak does not exist.

Next, since bank sequence N=2 is a check skip frame, the control circuit 210 executes control according to the information signal SPI at bank sequence N=0. That is, in the order of banks 0 to 1, the light-emitting element groups defined by banks 0 to 1 emit light in time series in synchronization with the output timing of the laser trigger signal. At this time, the light-receiving element groups defined by banks 0 to 1 are also driven in time series. On the other hand, based on the stop information included in the information signal SPI, the light-emitting element groups defined by banks 2 to 3 stop emitting light. At this time, the light-receiving element groups defined by banks 2 to 3 also stop driving. This makes it possible to stop driving the light-emitting element groups and light-receiving element groups corresponding to areas where object Obj does not exist, making it possible to save power. On the other hand, since the light-emitting element groups and light-receiving element groups corresponding to areas where object Obj was present continue to be driven, it is possible to determine whether object Obj continues to exist in the area where object Obj was present.

The histogram generating circuit 330a generates a histogram in time series corresponding to the light receiving element group defined by banks 0 to 1 under the control of the control circuit 330. That is, the control circuit 330 outputs an enable signal to the histogram generating circuit 330a as an enable signal corresponding to the light receiving element group defined by banks 0 to 1. On the other hand, the control circuit 330 outputs an disable signal to the histogram generating circuit 330a as an enable signal corresponding to the light receiving element group defined by banks 2 to 3.

Because the histograms corresponding to banks 0 to 1 are valid, the judgment circuit 330b judges whether or not each histogram has a predetermined peak, i.e., whether or not subject Obj is present. Because there is no reflected light from subject Obj in the area of the light receiving element corresponding to bank 0, the histogram corresponding to bank 0 does not generate a predetermined peak. For this reason, the judgment circuit 330b judges that there is no predetermined peak in each histogram corresponding to bank 0. On the other hand, there is reflected light from subject Obj in the area of the light receiving element corresponding to bank 1, so the histogram corresponding to bank 1 generates a predetermined peak. For this reason, the judgment circuit 330b judges that there is a predetermined peak in each histogram corresponding to bank 1.

Next, bank sequence N=3, like bank sequence N=0, is a check frame, and so is controlled in the same way as bank sequence N=0. In this way, a predetermined bank sequence becomes a check frame, and a predetermined number of bank sequences following the check frame become frames to be checked and skipped. As described above, in frames to be checked and skipped, the monitoring state based on the judgment of the check frame is reflected, and it becomes possible to stop driving the light-emitting element groups and light-receiving element groups corresponding to areas where no subject Obj is present, thereby enabling power saving.

In this way, the control circuit 330 outputs to the light emission control unit 200 an information signal SPI that controls the light emission method in the check skip frame (N = 1, 2, 4, 5, etc.) following the check frame based on the reaction state of the light receiving element 311 (see FIG. 4) in the check frame (N = 0, 3, etc.). This enables the light emission control unit 200 to efficiently emit light from the group of light emitting elements corresponding to the area in which the subject Obj is present, enabling power saving while suppressing a decrease in the tracking accuracy of the subject Obj.

[Second mode]
An example sequence of the second mode will be described with reference to Figures 10 and 11. As described above, the second mode is a firing limiting mode. In the processing example of Figures 10 and 11, a judgment process is performed based on histograms generated for each of banks 0 to 3. As will be described later, it is also possible to perform judgment processes for each of element groups MP0 to MP13 in parallel. For example, when performing judgment process for bank 0, it is also possible to perform judgment processes for each of element groups MP0, MP2, MP4, and MP5 included in bank 0 in parallel.

Figure 10 is a diagram for explaining the second mode. Figures 10(a) and 10(c) show a laser trigger signal in bank 0 in the second mode. The horizontal axis shows time ta, and the vertical axis shows the laser trigger signal. Figure 10(a) shows, for example, frame N=n, and Figure 9(c) shows, for example, frame N=n+1. Figures 10(b) and 10(d) show an example of a histogram in bank 0 in the second mode. The horizontal axis shows time ta, and the vertical axis corresponds to the occurrence frequency of the flight times of photons detected in element groups MP0, MP2, MP4, and MP5 measured by TDC 320. Figure 10(b) shows, for example, frame N=n, and Figure 10(d) shows, for example, frame N=n+1.

As shown in FIG. 10(b), for example, the determination circuit 330b (see FIG. 2) determines that the object Obj does not exist in the area corresponding to bank 0 in frame N=n because peak P10 does not exist. In this case, the control circuit 330 outputs an information signal SPI to the control circuit 210 via the stop information generation circuit 360, which limits the number of emissions from the light-emitting element group corresponding to bank 0 in frame N=n+1. In this way, the control circuit 330 outputs an information signal SPI to the control circuit 210, which is information that controls the light emission method of the light-emitting unit 100 in frames after N=n, based on the reaction state of the element groups MP0 to MP13. The light emission method in this case is to limit the number of emissions.

As shown in FIG. 10(c), this information signal SPI limits the number of laser shots from the light-emitting unit 100 in frame N=n+1. That is, the laser shot threshold control circuit 210b of the control circuit 210 limits the number of laser shots to the number of shots specified in the register 210a in response to the information signal SPI. That is, whereas the number of shots would normally correspond to time t10, it is limited to the number of shots corresponding to time t10a. In the second mode, although the number is limited, laser light is still emitted, so that the light-receiving element in the area corresponding to bank 0 is driven for the period corresponding to time t10a.

As a result, a histogram is generated as shown in FIG. 10(d). That is, the histogram generating circuit 330a can generate a histogram of the area corresponding to bank 0. Therefore, the judgment circuit 330b can judge whether or not an object is present. That is, in the second mode, the area where the object Obj is present is irradiated with laser light more times to judge whether or not an object is present. On the other hand, the number of laser light emissions is limited to areas where the object Obj is not present, enabling power saving. In addition, since the number of laser emissions is limited in mode 2, for example, the histogram generating circuit 330a normalizes the frequency to generate a histogram. That is, when the number of light receptions of the multiple light receiving elements 311 corresponding to bank 0 exceeds a predetermined threshold, the control circuit 330 limits the number of light emission emissions of the corresponding light emitting element 110m of the light emitting unit 100. In addition, the control circuit 330 executes control to limit the number of light emission emissions in a predetermined period for each area of the light emitting unit 100 corresponding to banks 0 to 3.

Figure 11 is a diagram showing a more detailed control sequence example of the second mode. Here, an example in which control is started from frame N=0 is described. As in Figure 9, the horizontal axis is time, and from the top, the number N of sequences (Bank sequences) that repeat banks 0 to 3, the type of bank, the schematic positions of the light-emitting unit 100 and the light-receiving elements and light-emitting elements of the light-detecting unit 300, the output timing of the laser trigger signal output by the laser trigger generating circuit 350, the output timing of the information signal SPI output by the stop information generating circuit 360, the light-emitting timing (laser) of the light-emitting element corresponding to each bank, the light-receiving state of the reflected light of the light-detecting unit 300 (SPAD reflected light state), the drive state (Enable) of the histogram generating circuit 330a, and the judgment state (peak detection) of the judgment circuit 330b are shown.

First, in bank sequence N=0, the light-emitting element groups defined by banks 0 to 3 emit light in chronological order in synchronization with the output timing of the laser trigger signal, in the order of banks 0 to 3. At the same time, the light-receiving element groups defined by banks 0 to 3 also emit light in chronological order.

The histogram generating circuit 330a generates histograms in time series corresponding to the light receiving element groups defined by banks 0 to 3. That is, the control circuit 330 outputs an enable signal to the histogram generating circuit 330a as an enable signal corresponding to the light receiving element groups defined by banks 0 to 3. Since all the histograms corresponding to banks 0 to 3 are valid, the determination circuit 330b determines whether or not a predetermined peak is present for each histogram, i.e., whether or not an object Obj is present. Since there is no reflected light from the object Obj in the light receiving element areas corresponding to banks 2 and 3, the predetermined peak is not generated in each of the histograms corresponding to banks 2 and 3. Therefore, the determination circuit 330b determines that there is no predetermined peak for each of the histograms corresponding to banks 2 and 3.

Then, the control circuit 330 outputs an information signal SPI, which includes information on the determination result of the determination circuit 330b, to the control circuit 210 between frames N=0 and N=1 via the stop information generation circuit 360. This information signal SPI includes information to limit the light emission of the light-emitting element groups corresponding to banks 2 and 3, since there is no specified peak in banks 2 and 3.

Next, in bank sequence N=1, the control circuit 210 executes control according to the information signal SPI in bank sequence N=0. That is, in the order of banks 0 to 1, the light-emitting element groups defined by banks 0 to 1 emit light in chronological order in synchronization with the output timing of the laser trigger signal. At this time, the light-receiving element groups defined by banks 0 to 1 are also driven in chronological order. Meanwhile, the number of light emitted by the light-emitting element groups defined by banks 2 to 3 is limited based on the limiting information included in the information signal SPI. At this time, the driving of the light-receiving element groups defined by banks 2 to 3 is also limited. This makes it possible to limit the driving of the light-emitting element groups and light-receiving element groups corresponding to areas where the subject Obj does not exist, enabling power saving.

The histogram generating circuit 330a generates a histogram in time series corresponding to the light receiving element groups defined by banks 0 to 3 under the control of the control circuit 330. That is, the control circuit 330 outputs an enable signal to the histogram generating circuit 330a as an enable signal corresponding to the light receiving element groups defined by banks 0 to 3.

Because the histograms corresponding to banks 0 to 3 are valid, the determination circuit 330b determines whether or not each histogram has a specified peak, i.e., whether or not subject Obj is present. Because there is no reflected light from subject Obj in the areas of the light receiving elements corresponding to banks 0 to 3, the histograms corresponding to banks 0 to 3 do not produce a specified peak. For this reason, the determination circuit 330b determines that the histograms corresponding to banks 0 to 3 do not have a specified peak.

Then, the control circuit 330 outputs an information signal SPI, which includes information on the determination result of the determination circuit 330b, to the control circuit 210 between frames N=1 and N=2 via the stop information generation circuit 360. This information signal SPI includes information to limit the light emission of the light-emitting element groups corresponding to banks 0 to 3, since there is no specified peak in banks 0 to 3.

Next, in bank sequence N=2, the control circuit 210 executes control according to the information signal SPI in bank sequence N=1. That is, in the order of banks 0 to 3, the light-emitting element groups defined by banks 0 to 3 emit light in time series in synchronization with the output timing of the laser trigger signal. At this time, the number of light emitted by the light-emitting element groups defined by banks 0 to 3 is limited based on the limiting information included in the information signal SPI. The driving time of the light-receiving element groups defined by banks 0 to 3 is also limited. This makes it possible to limit the driving of the light-emitting element groups and light-receiving element groups corresponding to areas where the subject Obj does not exist, making it possible to save power.

The histogram generating circuit 330a generates a histogram in time series corresponding to the light receiving element groups defined by banks 0 to 3 under the control of the control circuit 330. That is, the control circuit 330 outputs an enable signal to the histogram generating circuit 330a as an enable signal corresponding to the light receiving element groups defined by banks 0 to 3.

Because the histograms corresponding to banks 0 to 3 are valid, the determination circuit 330b determines whether or not each histogram has a specified peak, i.e., whether or not subject Obj is present. Because there is no reflected light from subject Obj in the areas of the light receiving elements corresponding to banks 0, 2, and 3, the histograms corresponding to banks 0, 2, and 3 do not produce a specified peak. For this reason, the determination circuit 330b determines that the histograms corresponding to banks 0, 2, and 3 do not have a specified peak.

On the other hand, since there is reflected light from the subject Obj in the area of the light receiving element corresponding to bank 1, a predetermined peak is generated in the histogram corresponding to bank 1. For this reason, the judgment circuit 330b judges that there is a predetermined peak in the histogram corresponding to bank 1. Then, the control circuit 330 outputs an information signal SPI including information on the judgment result of the judgment circuit 330b to the control circuit 210 between frames N=2 and N=3 via the stop information generation circuit 360. Since there is no predetermined peak in banks 0, 2, and 3, this information signal SPI includes information to limit the emission of the light emitting element groups corresponding to banks 0, 2, and 3.

Next, in bank sequence N=3, the control circuit 210 executes control according to the information signal SPI in bank sequence N=2. That is, in the order of banks 0 to 3, the light-emitting element groups defined by banks 0 to 3 emit light in time series in synchronization with the output timing of the laser trigger signal. At this time, the number of light-emitting element groups defined by banks 0, 2, and 3 is limited based on the limiting information included in the information signal SPI. The driving time of the light-receiving element groups defined by banks 0, 2, and 3 is also limited. On the other hand, the number of light-emitting element groups defined by bank 1 is not limited, and the driving time of the light-receiving element group defined by bank 1 is not limited. This makes it possible to limit the driving of the light-emitting element groups and light-receiving element groups corresponding to areas where the object Obj does not exist, making it possible to save power. On the other hand, the driving of the light-emitting element groups and light-receiving element groups corresponding to areas where the object Obj exists is not limited, making it possible to track with higher accuracy.

The histogram generating circuit 330a generates a histogram in time series corresponding to the light receiving element groups defined by banks 0 to 3 under the control of the control circuit 330. That is, the control circuit 330 outputs an enable signal to the histogram generating circuit 330a as an enable signal corresponding to the light receiving element groups defined by banks 0 to 3.

The determination circuit 330b determines whether or not a predetermined peak exists for each histogram, that is, whether or not an object Obj exists, since the histograms corresponding to banks 0 to 3 are valid. Since the light receiving element area corresponding to banks 0 to 3 contains reflected light from object Obj, a predetermined peak is generated in each histogram corresponding to banks 0 to 3. Therefore, the determination circuit 330b determines that each histogram corresponding to banks 0 to 3 has a predetermined peak. Then, the control circuit 330 outputs an information signal SPI including information on the determination result of the determination circuit 330b to the control circuit 210 between frames N=3 and N=4 via the stop information generation circuit 360. This information signal SPI includes information that does not limit the light emission of the light emitting element group corresponding to banks 0 to 3, since there is a predetermined peak in banks 0 to 3.

Next, in bank sequence N=4, the control circuit 210 executes control according to the information signal SPI in bank sequence N=3. That is, in the order of banks 0 to 3, the light-emitting element groups defined by banks 0 to 3 emit light in time series in synchronization with the output timing of the laser trigger signal. At this time, the restriction information included in the information signal SPI includes information indicating that there is no restriction. As a result, there is no restriction on the number of light-emitting element groups defined by banks 0 to 3, and there is no restriction on the light-receiving element groups defined by banks 0 to 3 or their drive time. As a result, there is no restriction on the drive of the light-emitting element groups and light-receiving element groups corresponding to the area where the subject Obj is present, allowing for more accurate tracking.

The histogram generating circuit 330a generates a histogram in time series corresponding to the light receiving element groups defined by banks 0 to 3 under the control of the control circuit 330. That is, the control circuit 330 outputs an enable signal to the histogram generating circuit 330a as an enable signal corresponding to the light receiving element groups defined by banks 0 to 3.

The determination circuit 330b determines whether or not a predetermined peak exists for each histogram, that is, whether or not an object Obj exists, since the histograms corresponding to banks 0 to 3 are valid. Since the light receiving element area corresponding to banks 0 to 3 contains reflected light from object Obj, a predetermined peak is generated in each histogram corresponding to banks 0 to 3. Therefore, the determination circuit 330b determines that each histogram corresponding to banks 0 to 3 has a predetermined peak. Then, the control circuit 330 outputs an information signal SPI including information on the determination result of the determination circuit 330b to the control circuit 210 between frames N=4 and N=5 via the stop information generation circuit 360. This information signal SPI includes information that does not limit the light emission of the light emitting element group corresponding to banks 0 to 3, since there is a predetermined peak in banks 0 to 3.

In this way, the control circuit 330 outputs to the light emission control unit 200 an information signal SPI that controls the light emission method in the bank sequence following the bank sequence based on the reaction state of the light receiving element 311 (see FIG. 4) in the bank sequence. This enables the light emission control unit 200 to efficiently emit light from the group of light emitting elements corresponding to the area in which the subject Obj is present, enabling power saving while suppressing a decrease in the tracking accuracy of the subject Obj.

Figure 12 is a diagram showing an example of the register configuration stored in register 340a of light detection unit 300. As shown in Figure 12 with reference to Figure 4, the register configuration has MPX (X coordinate), MPY (Y coordinate), and MPBM (BITMAP) for each element group MP0-13 of Banks 0-3. MPX (X coordinate) and MPY (Y coordinate) indicate the range of light receiving elements 311. MPBM (BITMAP) is information indicating whether or not it is included in Banks 0-3. By referring to this MPX (X coordinate), MPY (Y coordinate), and MPBM (BITMAP) information, the control circuit 340 can control the light receiving elements 311 to be driven for each of Banks 0-3.

Figure 13 is a diagram showing an example of the data configuration of the information signal SPI generated by the stop information generating circuit 360. For example, information on one of banks 0 to 3 is assigned to BankID in 2 bits. For example, information on one of the corresponding MPs 0 to 13 is assigned to MP number in 4 bits. For example, a code indicating whether or not laser stop threshold control is performed is assigned to Enable in 1 bit. For example, if Enable is 0, it is the first mode (normal), and if Enable is 1, it is the second mode (stop threshold control). These make up the number of banks 0 to 3 x the number of element groups MP0 to 13 = 56. In the laser emission number threshold field, for example, 0 is specified in 30 bits for the first mode, and for example, the number of times the laser is emitted is specified in 30 bits for the second mode.

[AP recovery process]
In this embodiment, when it is determined that the object Obj is present, the control circuit 340 returns the distance measurement system 1 from the monitoring mode to the normal mode according to a predetermined condition. That is, in this embodiment, the histogram generation circuit 330a adds up the number of times that each of the predetermined light receiving elements 311 receives light in accordance with the elapsed time from the timing of irradiation of the irradiation light in the light emitting unit 100, and the control circuit 340 transmits a return signal to the main control unit 500 when the added number (frequency) exceeds a predetermined threshold. Thus, in this embodiment, the control circuit 340 transmits a return signal to the main control unit 500 in accordance with a state in which the peak in the histogram generated for each of the element groups MP0 to MP13, for example, exceeds a predetermined threshold within a predetermined period.

The main control unit 500 may be configured with an AP recovery notification register. In this case, the control circuit 340 may access the AP recovery notification register via the external interface 370, for example using I2C, and notify information for controlling the light emission method. In this case, the main control unit 500 is in a power-saving state in the monitoring mode, but I2C communication is maintained in a state where it is possible via the external interface 370. This allows the main control unit 500 to periodically read the AP recovery notification register to observe whether the AP has recovered. As can be seen from this, it is possible to return the main control unit 500 to normal mode while maintaining power saving.

In normal mode, as described above, the imaging unit 400 and the main control unit 500 return to their normal monitoring state. For example, in normal mode, as described above, it becomes possible for the imaging unit 400 to capture two-dimensional images, and to synthesize two-dimensional images with distance images.

The details of the restoration process example in this embodiment will be described below with reference to Figures 14 to 17. Figure 14 is a diagram showing an example of a histogram generated by the histogram generating circuit 330a for each of the element groups MP0 to MP13 in frames N = 0 to 3. For example, Figure 14 (a) is a histogram for frame N = 0 of the element group MP0, Figure 14 (b) is a histogram for frame N = 1 of the element group MP0, Figure 14 (c) is a histogram for frame N = 2 of the element group MP0, and Figure 14 (d) is a histogram for frame N = 3 of the element group MP0. The judgment circuit 330b judges whether the peak P10 exceeds the threshold value Cth, and records the number of the frame in which it exceeds the threshold value Cth and the information of the element groups MP0 to MP13 in the register 340a. For example, the judgment circuit 330b judges that a peak exists when the number of times the threshold value Cth is exceeded in succession exceeds a predetermined number. This allows the determination circuit 330b to determine that an object Obj is present when the object Obj is continuously present in the monitoring area. In other words, the determination circuit 330b does not determine that an object Obj is present when the passing speed of the object Obj is faster than the expected monitoring target, such as when a bird or the like happens to pass by. This prevents erroneous determinations due to objects Obj that are not the monitoring target.

Similarly, FIG. 15 shows an example of histograms generated by the histogram generating circuit 330a for each of the element groups MP0 to MP13 in frames N=0 to 3. For example, FIG. 15(a) is a histogram for frame N=0 of the element group MP3, FIG. 15(b) is a histogram for frame N=1 of the element group MP3, FIG. 15(c) is a histogram for frame N=2 of the element group MP3, and FIG. 15(d) is a histogram for frame N=3 of the element group MP3. The determination circuit 330b determines whether the peak P10 exceeds the threshold Cth, and records the number of the frame in which it exceeds the threshold Cth and information on the element groups MP0 to MP13 in the register 340a. For example, the determination circuit 330b determines that a peak exists because the number of consecutive times the threshold Cth is exceeded exceeds a predetermined number (for example, 3). In other words, it determines that the object Obj exists in the corresponding area of the element group MP3.

Figure 16 is a diagram showing an example of determination information recorded in register 340a. The horizontal axis indicates frames N=0 to 4, and the vertical axis indicates element groups MP0 to MP13. In Figure 15, element groups MP0 to MP13 that are determined to have a peak are indicated with a circle, and element groups MP0 to MP13 that are not determined to have a peak are indicated with an x. For example, an AP recovery threshold Thap=6 is set in advance in register 340a. In the example of Figure 16, the number of circles is 8, which exceeds the AP recovery threshold Thap=6, so the control circuit 340 transmits a recovery signal to the main control unit 500 to return the ranging system 1 from monitoring mode to normal mode.

Figure 17 is a diagram showing an example of non-recovery determination information recorded in register 340a. The horizontal axis indicates frames N=0-4, and the vertical axis indicates element groups MP0-MP13. In Figure 16, element groups MP0-MP13 that are determined to have a peak are indicated with a circle, and element groups MP0-MP13 that are not determined to have a peak are indicated with an x. In the example of Figure 16, the number of circles is four, which does not exceed the AP recovery threshold Thap=6, so the control circuit 340 maintains the monitoring mode. In other words, the control circuit 340 does not send a recovery signal to the main control unit 500.

Figure 18 is a diagram showing an example of a sequence including AP recovery processing. The horizontal axis is time, and from the top, it shows the overall control mode (State) indicating whether it is monitoring mode (dToF-ROI) or normal mode (AP recovery), the control mode within the monitoring mode, the code indicating the AP recovery judgment result, the number N of times the sequence (Bank sequence) repeats banks 0 to 3, the type of bank, the output timing of the laser trigger signal output by the laser trigger generation circuit 350, the output timing of the information signal SPI output by the stop information generation circuit 360, the AP recovery notification signal output by the control circuit 330, the emission timing when the emission is stopped (Laser (1)), and the emission timing when the emission is not stopped (Laser (2)).

First, in bank sequence N=n, the distance measurement system 1 is controlled in monitoring mode (dToF-ROI). The control mode at this time is the first mode (MODE=thinning mode), and the number of thinning DEC is 3. In other words, this is an example in which one check frame (N=n) is followed by two check skip frames. For example, the determination circuit 330b determines that the object Obj exists in the area corresponding to banks 0 to 1, and determines that the object Obj does not exist in the area corresponding to banks 2 to 3. As a result, the control circuit 330 outputs an information signal SPI including information on the determination result of the determination circuit 330b to the control circuit 210 between frames N=n and N=n+1 via the stop information generation circuit 360. This information signal SPI includes information to stop the emission of the light-emitting element groups corresponding to banks 2 and 3 because there is no specified peak in banks 2 and 3.

Next, in bank sequence N=n+1, since it is a check skip frame, the control circuit 210 executes control according to the information signal SPI in bank sequence N=n. That is, in the order of banks 0 to 1, the light-emitting element groups defined by banks 0 to 1 emit light in chronological order, synchronizing with the output timing of the laser trigger signal. At this time, the light-receiving element groups defined by banks 0 to 1 are also driven in chronological order. Meanwhile, based on the stop information included in the information signal SPI, the light-emitting element groups defined by banks 2 to 3 stop emitting light. At this time, the light-receiving element groups defined by banks 2 to 3 also stop driving.

On the other hand, when bank sequence N=n+1 ends, for example, as shown in FIG. 16, in four consecutive frames N=n-2, n-1, n, n+1, the element group MP0 to MP13 determined to have a peak exceeds the threshold value of 6, so the control circuit 330 executes control for normal mode (AP return). At this time, the information signal SPI also stops, so the drive control state of the light-emitting element 110m of bank sequence N=n+1 and the light-receiving element 311 is maintained. For example, in the example of (Laser (1)), the emission of the light-emitting element 110m corresponding to banks 2 and 3 is stopped. For example, in the example of (Laser (2)), there is no light-emitting element 110 that is stopped.

In this way, even when the control of the normal mode (AP return) is executed, the driving of the light-emitting element group and the light-receiving element group corresponding to the area where the object Obj exists is not restricted, so more accurate tracking is possible. On the other hand, the driving of the light-emitting element group and the light-receiving element group corresponding to the area where the object Obj does not exist is stopped, so power saving is possible. In this way, even when the control of the normal mode (AP return) is executed, the control circuit 330 outputs an information signal SPI to the light emission control unit 200, which controls the light emission method in the bank sequence following the bank sequence, based on the reaction state of the light-receiving element in the bank sequence. As a result, even when the control of the normal mode (AP return) is executed, the driving of the light-emitting element group and the light-receiving element group corresponding to the area where the object Obj exists is not restricted, so more accurate tracking is possible. On the other hand, the driving of the light-emitting element group and the light-receiving element group corresponding to the area where the object Obj does not exist is stopped, so power saving is possible.

Figure 19 is a flowchart showing an example of processing according to this embodiment. First, the operator sets the control parameters in the register 340a of the control circuit 330 via the operation unit of the light detection unit 300 (step S100). That is, in the register 340a, it is set whether the control mode is the frame thinning mode (first mode) or the number of shots limited mode (second mode). In the frame thinning mode, the number of thinnings DEC is set. The number of thinnings DEC-1 corresponds to the number of check skip frames. Furthermore, the laser number threshold Th is set in the register 210a. In this example, the laser number threshold Th in the first mode is 0. Here, in the first frame N=0 without the information signal SPI, control is performed with the default setting.

Next, the control circuit 330 sets the number of bank sequences N=0 (step S102), sets the bank number NB=0, and starts controlling the bank sequence (step S104). Next, the control circuit 330 judges whether a predetermined bank sequence has been completed (step S106). For example, if the bank number NB=0 to 3, it is judged that the bank sequence has not been completed (No in step S106), and the laser firing threshold control circuit 210b controls the number of laser firings of bank NB of the bank sequence according to the laser firing threshold Th (step S108). That is, in the frame thinning mode, 0 or a default setting value is set as the number of laser firings according to the information signal SPI. On the other hand, in the firing limit mode, the laser firing threshold Th or a default setting value is set as the number of laser firings. As mentioned above, when N=0, the number of firings is the default setting value.

Next, the histogram generation circuit 330a generates a histogram of the area corresponding to bank NB, and the judgment circuit 330b detects a predetermined peak based on the histogram (step S110). If the predetermined peak is not detected, the judgment circuit 330b stores the address information of the area corresponding to bank NB in the register 340a (step S112). The control circuit 330 adds 1 to the bank number NB (step S114). Then, the process from step S106 is repeated, and the control circuit 330 judges that the bank sequence has ended if, for example, bank number NB = 4 (YES in step S106).

Next, the control circuit 330 determines whether the control mode is the frame thinning mode or the number limiting mode (step S116). If the control mode is the frame thinning mode (Yes in step S116), the control circuit 330 determines the number of bank sequences N (step S118).

If the number of bank sequences N=0, then, as control for the check frame, the control circuit 330 sets the address of the light emitting element 110m that stops the laser in the information signal SPI according to the information of step S114 (step S122), generates an information signal SPI including a laser firing value setting of 0 in the stop information generating circuit 360, and outputs it to the laser firing threshold control circuit 210b (step S122). Next, according to the information of step S114, the element group corresponding to the bank that stops driving is stored in the register 340a from among the element groups MP0 to MP13 (step S124), and the control circuit 330 sets the number of bank sequences N=N+1 (step S126), and repeats the process from step S106.

On the other hand, if the number of bank sequences N is 1 to (DEC-2), it is a check skip frame, so the control state is not changed and the process from (step S126) is repeated. On the other hand, if the number of bank sequences N = (DEC-2), it is the final check skip frame, so control as a check frame is started in the next frame, so the stop address setting of the light-emitting element is turned off (OFF) (step S128), the stop address setting of element groups MP0 to MP13 is turned off (OFF) (step S130), and the process from step S126 is repeated.

On the other hand, if the control mode is the shot count limit mode (No in step S116), the control circuit 330 sets the address of the light-emitting element 110m that stops the laser in the information signal SPI (step S132), and causes the stop information generation circuit 360 to generate an information signal SPI including the laser shot count setting = Th, and outputs it to the laser shot count threshold control circuit 210b (step S134). Then, the process from step S126 is repeated.

As described above, according to this embodiment, the control circuit 330 outputs to the light emission control unit 200 an information signal SPI that controls the light emission method in the bank sequence following the bank sequence, based on the reaction state of the light receiving elements in the bank sequence. This allows for more accurate tracking of the subject Obj, as the driving of the light emitting element group and light receiving element group corresponding to the area where the subject Obj is present is not restricted. On the other hand, the driving of the light emitting element group and light receiving element group corresponding to the area where the subject Obj is not present is stopped or restricted, making it possible to save power.

(Modification 1 of the first embodiment)
The ranging system 1 according to the first modification of the first embodiment differs from the ranging system 1 according to the first embodiment in that the range of judgment of the histogram generated by the histogram generating circuit 330a is changed according to the distance range of the object Obj to be monitored. The differences from the ranging system 1 according to the first embodiment will be described below.

In the distance measurement system 1 according to the first modified example of the first embodiment, the peak detection range is set in the register 340a of the control circuit 330 via the operation unit of the light detection unit 300. FIG. 20 shows the peak detection range tn0 in the case of a short distance. FIGS. 20(a) and 20(b) show histograms generated by the histogram generation circuit 330a. The horizontal axis represents time, and the vertical axis represents frequency. FIG. 20(a) shows a case where there is no peak in the peak detection range tn0, and FIG. 20(b) shows a case where there is a peak in the peak detection range tn0. The determination circuit 330b determines whether or not there is a peak from the peak detection range tn0. The other processes are the same as those of the distance measurement system 1 according to the first embodiment.

Figure 21 shows the peak detection range tf0 in the case of a long distance. Figures 21(a) and (b) show histograms generated by the histogram generation circuit 330a. The horizontal axis is time, and the vertical axis is frequency. Figure 21(a) shows a case where there is no peak in the peak detection range tf0, and Figure 21(b) shows a case where there is a peak in the peak detection range tf0. The judgment circuit 330b judges the presence or absence of a peak from the peak detection range tf0. In other words, the frequency calculation by adding up the number of times light is received in the light receiving unit 100 is set to a predetermined time range from the timing of irradiation of the light emitted by the light emitting unit 110. These time ranges are set according to the distance measurement range. Other processing is equivalent to that of the distance measurement system 1 according to the first embodiment.

As described above, in the distance measurement system 1 according to the first modified example of the first embodiment, the detection range of the peak in the histogram is set according to the distance range of the subject Obj to be monitored. This makes it possible to determine whether or not the subject Obj in the desired distance range is present in the monitoring area.

(Modification 2 of the First Embodiment)
The ranging system 1 according to the modified example 2 of the first embodiment differs from the ranging system 1 according to the modified example 1 of the first embodiment in that, when it is determined that the subject Obj to be monitored does not exist, the laser intensity of the light emitting unit 100 can be changed. The differences from the ranging system 1 according to the modified example 1 of the first embodiment will be described below.

Figure 22 is a diagram showing an example of intensity change in the case of a long distance. Figures 22(a) and (b) show histograms generated by the histogram generation circuit 330a. The horizontal axis represents time, and the vertical axis represents frequency. Figure 22(a) shows a case where there is no peak in the peak detection range tf0, and Figure 22(b) shows an example where control circuit 210 is controlled to increase the laser intensity according to the control of control circuit 340. By increasing the laser intensity, judgment circuit 330b can determine that there is a peak in peak detection range tf0. Other processing is equivalent to that of distance measurement system 1 according to variant 1 of the first embodiment.

As described above, in the distance measurement system 1 according to the second modification of the first embodiment, when it is determined that the object Obj to be monitored is not present, the laser intensity of the light-emitting unit 100 can be changed. This makes it possible to detect reflected light from an object Obj with low reflectance, and to determine whether or not an object Obj with low reflectance is present in the monitoring area.

(Modification 3 of the first embodiment)
The ranging system 1 according to the third modification of the first embodiment differs from the ranging system 1 according to the first modification of the first embodiment in the determination process of the processing circuit 330. The differences from the ranging system 1 according to the first modification of the first embodiment will be described below.

Figure 23 is a diagram showing the determination process of the processing circuit 330. The control circuit 340 causes the light-emitting unit 100 to emit light in frame N=0. In this case, it is possible to cause the light-emitting element 110m corresponding to any of banks 0 to 3 to emit light. The processing circuit 330 acquires a pulse (detection signal PXOUT) whose rising edge occurs when each SPAD element detects a photon from the corresponding element group in banks 0 to 3, and sets the total number of detection signals PXOUT as count value A [count]. On the other hand, the processing circuit 330 acquires a pulse (detection signal PXOUT) whose rising edge occurs when each SPAD element detects a photon with the light-emitting unit 100 stopped from the corresponding element group in banks 0 to 3, and sets the total number of detection signals PXOUT as count value B [count].

The processing circuit 330 subtracts a count value B [count] corresponding to the ambient light from the count value A [count], and if it exceeds a predetermined value, it determines that a subject is present in the area corresponding to the group of elements to be monitored in banks 0 to 3. All other processing is the same as in the ranging system 1 according to the first embodiment. In this way, with the effects of ambient light reduced, the processing circuit 330 can determine whether or not a subject is present in the monitored area.

<<Application Examples>>
The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as a part mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, a construction machine, or an agricultural machine (tractor).

24 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile control system to which the technology disclosed herein can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. In the example shown in FIG. 24, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside vehicle information detection unit 7400, an inside vehicle information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these multiple control units may be, for example, an in-vehicle communication network conforming to any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay (registered trademark).

Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a storage unit that stores the programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various control target parts. Each control unit includes a network I/F for communicating with other control units via a communication network 7010, and a communication I/F for communicating with parts or sensors inside and outside the vehicle by wired communication or wireless communication. In FIG. 24, the functional configuration of the integrated control unit 7600 includes a microcomputer 7610, a general-purpose communication I/F 7620, a dedicated communication I/F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I/F 7660, an audio/image output unit 7670, an in-vehicle network I/F 7680, and a storage unit 7690. Other control units also include a microcomputer, a communication I/F, a storage unit, and the like.

The drive system control unit 7100 controls the operation of parts related to the drive system of the vehicle according to various programs. For example, the drive system control unit 7100 functions as a control unit for a drive force generating unit for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking unit for generating a braking force of the vehicle. The drive system control unit 7100 may also function as a control unit for an ABS (Antilock Brake System) or an ESC (Electronic Stability Control), etc.

The drive system control unit 7100 is connected to a vehicle state detection unit 7110. The vehicle state detection unit 7110 includes at least one of a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor for detecting the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine speed, or the rotation speed of the wheels, for example. The drive system control unit 7100 performs arithmetic processing using the signal input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering unit, the brake unit, etc.

The body system control unit 7200 controls the operation of various parts installed in the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control unit for a keyless entry system, a smart key system, a power window section, or various lamps such as headlamps, tail lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals from various switches transmitted from a portable device that replaces a key can be input to the body system control unit 7200. The body system control unit 7200 receives these radio waves or signals and controls the door lock section, power window section, lamps, etc. of the vehicle.

The battery control unit 7300 controls the secondary battery 7310, which is the power supply source for the drive motor, according to various programs. For example, information such as the battery temperature, battery output voltage, or remaining capacity of the battery is input to the battery control unit 7300 from the battery section equipped with the secondary battery 7310. The battery control unit 7300 performs calculations using these signals, and controls the temperature regulation of the secondary battery 7310 or the cooling section equipped in the battery section.

The outside-vehicle information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of the imaging unit 7410 and the outside-vehicle information detection unit 7420 is connected to the outside-vehicle information detection unit 7400. The imaging unit 7410 includes at least one of a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside-vehicle information detection unit 7420 includes at least one of, for example, an environmental sensor for detecting the current weather or climate, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, etc., around the vehicle equipped with the vehicle control system 7000.

The environmental sensor may be, for example, at least one of a raindrop sensor that detects rain, a fog sensor that detects fog, a sunshine sensor that detects the degree of sunshine, and a snow sensor that detects snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar unit, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) unit. The imaging unit 7410 and the outside vehicle information detection unit 7420 may each be provided as an independent sensor or unit, or may be provided as a unit in which multiple sensors or units are integrated.

25 shows an example of the installation positions of the imaging unit 7410 and the outside vehicle information detection unit 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are provided, for example, at least one of the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle cabin of the vehicle 7900. The imaging unit 7910 provided on the front nose and the imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 7900. The imaging units 7912 and 7914 provided on the side mirrors mainly acquire images of the sides of the vehicle 7900. The imaging unit 7916 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918, which is installed on the top of the windshield inside the vehicle, is primarily used to detect preceding vehicles, pedestrians, obstacles, traffic signals, traffic signs, lanes, etc.

In addition, FIG. 25 shows an example of the imaging ranges of each of the imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose, imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided on the side mirrors, respectively, and imaging range d indicates the imaging range of the imaging unit 7916 provided on the rear bumper or back door. For example, an overhead image of the vehicle 7900 viewed from above is obtained by superimposing the image data captured by the imaging units 7910, 7912, 7914, and 7916.

The outside information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided on the front, rear, sides, corners, and upper part of the windshield inside the vehicle 7900 may be, for example, ultrasonic sensors or radar units. The outside information detection units 7920, 7926, and 7930 provided on the front nose, rear bumper, back door, and upper part of the windshield inside the vehicle 7900 may be, for example, LIDAR units. These outside information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, etc.

Returning to FIG. 24, the explanation will be continued. The outside-vehicle information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle and receives the captured image data. The outside-vehicle information detection unit 7400 also receives detection information from the connected outside-vehicle information detection unit 7420. If the outside-vehicle information detection unit 7420 is an ultrasonic sensor, a radar unit, or a LIDAR unit, the outside-vehicle information detection unit 7400 transmits ultrasonic waves or electromagnetic waves, and receives information on the received reflected waves. The outside-vehicle information detection unit 7400 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, or characters on the road surface, based on the received information. The outside-vehicle information detection unit 7400 may perform environmental recognition processing for recognizing rainfall, fog, road surface conditions, etc., based on the received information. The outside-vehicle information detection unit 7400 may calculate the distance to an object outside the vehicle based on the received information.

The outside vehicle information detection unit 7400 may also perform image recognition processing or distance detection processing to recognize people, cars, obstacles, signs, or characters on the road surface based on the received image data. The outside vehicle information detection unit 7400 may perform processing such as distortion correction or alignment on the received image data, and may also generate an overhead image or a panoramic image by synthesizing image data captured by different imaging units 7410. The outside vehicle information detection unit 7400 may also perform viewpoint conversion processing using image data captured by different imaging units 7410.

The in-vehicle information detection unit 7500 detects information inside the vehicle. For example, a driver state detection unit 7510 that detects the state of the driver is connected to the in-vehicle information detection unit 7500. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biometric information, or a microphone that collects sound inside the vehicle. The biosensor is provided, for example, on the seat or steering wheel, and detects the biometric information of a passenger sitting in the seat or a driver holding the steering wheel. The in-vehicle information detection unit 7500 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing. The in-vehicle information detection unit 7500 may perform processing such as noise canceling processing on the collected sound signal.

The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. The integrated control unit 7600 is connected to an input unit 7800. The input unit 7800 is realized by a unit that can be operated by the passenger, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by voice recognition of a voice input by a microphone may be input to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control unit using infrared or other radio waves, or an external connection device such as a mobile phone or a PDA (Personal Digital Assistant) that supports the operation of the vehicle control system 7000. The input unit 7800 may be, for example, a camera, in which case the passenger can input information by gestures. Alternatively, data obtained by detecting the movement of a wearable unit worn by the passenger may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by a passenger or the like using the input unit 7800 and outputs the signal to the integrated control unit 7600. The passenger or the like operates the input unit 7800 to input various data to the vehicle control system 7000 and to instruct processing operations.

The storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc. The storage unit 7690 may also be realized by a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device, etc.

The general-purpose communication I/F 7620 is a general-purpose communication I/F that mediates communication between various devices present in the external environment 7750. The general-purpose communication I/F 7620 may implement cellular communication protocols such as GSM (registered trademark) (Global System of Mobile communications), WiMAX (registered trademark), LTE (registered trademark) (Long Term Evolution) or LTE-A (LTE-Advanced), or other wireless communication protocols such as wireless LAN (also called Wi-Fi (registered trademark)) and Bluetooth (registered trademark). The general-purpose communication I/F 7620 may connect to devices (e.g., application servers or control servers) present on an external network (e.g., the Internet, a cloud network, or an operator-specific network) via, for example, a base station or an access point. In addition, the general-purpose communication I/F 7620 may connect to a terminal located near the vehicle (e.g., a driver's, pedestrian's, or store's terminal, or a machine type communication (MTC) terminal) using, for example, P2P (Peer To Peer) technology.

The dedicated communication I/F 7630 is a communication I/F that supports a communication protocol developed for use in a vehicle. The dedicated communication I/F 7630 may implement a standard protocol such as WAVE (Wireless Access in Vehicle Environment), which is a combination of the lower layer IEEE 802.11p and the upper layer IEEE 1609, DSRC (Dedicated Short Range Communications), or a cellular communication protocol. The dedicated communication I/F 7630 performs V2X communication, which is a concept including one or more of, for example, vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

The positioning unit 7640 performs positioning by receiving, for example, GNSS signals from GNSS (Global Navigation Satellite System) satellites (for example, GPS signals from GPS (Global Positioning System) satellites) and generates position information including the latitude, longitude, and altitude of the vehicle. The positioning unit 7640 may determine the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, PHS, or smartphone that has a positioning function.

The beacon receiver 7650 receives, for example, radio waves or electromagnetic waves transmitted from radio stations installed on the road, and acquires information such as the current location, congestion, road closures, and travel time. The functions of the beacon receiver 7650 may be included in the dedicated communication I/F 7630 described above.

The in-vehicle device I/F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I/F 7660 may establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB). The in-vehicle device I/F 7660 may also establish a wired connection such as USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), or MHL (Mobile High-Definition Link) via a connection terminal (and a cable, if necessary) not shown. The in-vehicle device 7760 may include, for example, at least one of a mobile device or wearable device owned by the passenger, or an information device carried into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation unit that searches for a route to an arbitrary destination. The in-vehicle device I/F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

The in-vehicle network I/F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I/F 7680 transmits and receives signals, etc., in accordance with a specific protocol supported by the communication network 7010.

The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information acquired through at least one of the general-purpose communication I/F 7620, the dedicated communication I/F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I/F 7660, and the in-vehicle network I/F 7680. For example, the microcomputer 7610 may calculate the control target value of the driving force generating unit, the steering mechanism, or the braking unit based on the acquired information inside and outside the vehicle, and output a control command to the drive system control unit 7100. For example, the microcomputer 7610 may perform cooperative control for the purpose of realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, vehicle speed maintenance driving, vehicle collision warning, or vehicle lane departure warning. In addition, the microcomputer 7610 may control the driving force generating unit, steering mechanism, braking unit, etc. based on the acquired information about the surroundings of the vehicle, thereby performing cooperative control for the purpose of automatic driving, which allows the vehicle to travel autonomously without relying on the driver's operation.

The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people based on information acquired through at least one of the general-purpose communication I/F 7620, the dedicated communication I/F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I/F 7660, and the in-vehicle network I/F 7680, and may generate local map information including information about the surroundings of the vehicle's current position. The microcomputer 7610 may also predict dangers such as vehicle collisions, the approach of pedestrians, or entry into closed roads based on the acquired information, and generate warning signals. The warning signals may be, for example, signals for generating warning sounds or turning on warning lamps.

The audio/image output unit 7670 transmits at least one output signal of audio and image to an output unit capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of FIG. 24, an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as output units. The display unit 7720 may include, for example, at least one of an on-board display and a head-up display. The display unit 7720 may have an AR (Augmented Reality) display function. The output unit may be other units such as headphones, a wearable device such as a glasses-type display worn by the passenger, a projector, or a lamp, other than these units. When the output unit is a display unit, the display unit visually displays the results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats such as text, images, tables, graphs, etc. Furthermore, if the output unit is an audio output unit, the audio output unit converts an audio signal consisting of reproduced voice data or acoustic data, etc., into an analog signal and outputs it audibly.

24, at least two control units connected via the communication network 7010 may be integrated into one control unit. Alternatively, each control unit may be composed of multiple control units. Furthermore, the vehicle control system 7000 may include another control unit not shown. In addition, some or all of the functions performed by any control unit in the above description may be provided by another control unit. In other words, as long as information is transmitted and received via the communication network 7010, a predetermined calculation process may be performed by any control unit. Similarly, a sensor or part connected to any control unit may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.

A computer program for implementing each function of the distance measuring system 1 according to this embodiment described with reference to FIG. 1 can be implemented in any control unit or the like. A computer-readable recording medium on which such a computer program is stored can also be provided. The recording medium can be, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a flash memory. The computer program can also be distributed, for example, via a network, without using a recording medium.

In the vehicle control system 7000 described above, the distance measurement system 1 according to this embodiment described using FIG. 1 can be applied to the integrated control unit 7600 of the application example shown in FIG. 24. For example, the distance measurement system 1 corresponds to the imaging unit 7410.

This technology can be configured as follows:

(1)
a light receiving section having a plurality of light receiving elements that receive light reflected by an object;
a control unit that transmits an information signal including information for controlling a light emission method in frames N and thereafter based on a reaction state of the plurality of light receiving elements in the Nth frame to an irradiation control unit that controls a light emitting unit that emits the irradiation light;
A light detection element comprising:

(2)
The light detection element according to (1), wherein the light emission method is to limit the number of emitted lights.

(3)
The light detection element according to (2), wherein the control circuit limits the number of light emission for each region of the light-emitting portion.

(4)
The light detection element according to (1), wherein the light emission method is to stop the light emission for a predetermined period of time.

(5)
The light detection element according to (4), wherein the control circuit stops the light emission for each region of the light-emitting portion.

(6)
The irradiation light is irradiated a predetermined number of times,
The light detection element according to (1), wherein when the number of times the plurality of light receiving elements receive light exceeds a predetermined threshold, the control circuit outputs a recovery signal to a main control unit.

(7)
The light detection element according to (6), wherein the number of times of receiving light is set within a predetermined time range from the irradiation timing of the irradiation light.

(8)
The light detection element according to (7), wherein the predetermined time range is set according to a distance measurement range.

(9)
The light detection element according to (1), wherein the irradiated light is laser light, and the reflected light is photons.

(10)
The light emitting unit has a plurality of light emitting elements,
a light receiving surface of the light receiving unit is divided into a plurality of element group regions,
the plurality of light-emitting elements are associated with the plurality of element group regions based on their positions;
The light detection element according to (1), wherein the number of emitted lights is limited for each of the light-emitting elements associated with the plurality of element group regions.

(11)
The light detection element according to (10), wherein the control circuit limits the number of emissions for each of the elements associated with the plurality of element group regions based on a reaction state for each of the plurality of element group regions.

(12)
The light emitting unit has a plurality of light emitting elements,
a light receiving surface of the light receiving unit is divided into a plurality of element group regions,
the plurality of light-emitting elements are associated with the plurality of element group regions based on their positions;
The light detection element according to (1), wherein the light emission is stopped for each element associated with the plurality of element group regions.

(13)
The light detection element according to (12), wherein the control circuit stops light emission for each of the elements associated with the plurality of element group regions based on a reaction state for each of the plurality of element group regions.

(14)
The irradiation light is irradiated a predetermined number of times,
The light detection element according to (1), wherein when the number of times the plurality of light receiving elements receive light exceeds a predetermined threshold, the control circuit outputs a recovery signal to a main control unit.

(15)
The irradiation light is irradiated a predetermined number of times,
adding up the number of times that the light is received by each of the plurality of light receiving elements in accordance with the elapsed time from the irradiation timing of the irradiation light;
When the added number exceeds a predetermined threshold value,
The light detection element according to (2), wherein the control circuit limits the number of emitted lights.

(16)
The irradiation light is irradiated a predetermined number of times,
adding up the number of times that the light is received by each of the plurality of light receiving elements in accordance with the elapsed time from the irradiation timing of the irradiation light;
When the added number exceeds a predetermined threshold value,
The light detection element according to (4), wherein the control circuit stops the light emission for a predetermined period.

(17)
The light detection element according to (1), wherein the frame corresponds to a predetermined control period.

(18)
A step of receiving light reflected by an object by a plurality of light receiving elements;
a control step of transmitting an information signal including information for controlling a light emission method in frames N and thereafter based on a reaction state of the plurality of light receiving elements in the Nth frame to an irradiation control unit that controls a light emitting unit that emits the irradiation light;
A distance measuring method comprising:

(19)
a light-emitting unit having a light-emitting portion that emits irradiation light;
an illumination control unit that controls the light emitting unit;
A light detection unit;
A ranging system comprising:
The light detection unit includes:
a light receiving section having a plurality of light receiving elements that receive light reflected by an object;
a control circuit that transmits an information signal including information for controlling a light emission method in frames N and thereafter based on a reaction state of the plurality of light receiving elements in the Nth frame to an irradiation control unit that controls a light emitting unit that emits the irradiation light;
A ranging system comprising:

(20)
A main control unit capable of controlling the light detection unit,
The distance measuring system according to (19), wherein the main control unit is in a power saving state when the light receiving unit controls the light emission method.

The aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that may be conceived by a person skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. In other words, various additions, modifications, and partial deletions are possible within the scope that does not deviate from the conceptual idea and intent of the present disclosure derived from the contents defined in the claims and their equivalents.

1: Distance measurement system, 100: Light emitter, 200: Light emitter control unit, 300: Light detector (light detector element), 310: Light receiver, 340: Control circuit, 500: Main control unit.

Claims (20)

a light receiving section having a plurality of light receiving elements that receive light reflected by an object;
a control circuit that transmits an information signal including information for controlling a light emission method in frames N and thereafter based on a reaction state of the plurality of light receiving elements in the Nth frame to an irradiation control unit that controls a light emitting unit that emits the irradiation light;
A light detection element comprising: The light detection element according to claim 1, wherein the light emission method is to limit the number of emitted lights. The light detection element according to claim 2, wherein the control circuit limits the number of light emitted for each region of the light-emitting portion. The light detection element according to claim 1, wherein the light emission method is to stop the light emission for a predetermined period of time. The light detection element according to claim 4, wherein the control circuit stops the emission of light for each region of the light-emitting portion. The irradiation light is irradiated a predetermined number of times,
The light detection element according to claim 1 , wherein when the number of times that the plurality of light receiving elements receive light exceeds a predetermined threshold, the control circuit outputs a recovery signal to a main control unit. The light detection element according to claim 6, wherein the number of times the light is received is set within a predetermined time range from the timing of irradiation of the irradiation light. The light detection element according to claim 7, wherein the predetermined time range is set according to a distance measurement range. The light detection element of claim 1, wherein the irradiated light is laser light and the reflected light is photons. The light emitting unit has a plurality of light emitting elements,
a light receiving surface of the light receiving unit is divided into a plurality of element group regions,
the plurality of light-emitting elements are associated with the plurality of element group regions based on their positions;
The light detection element according to claim 1 , wherein the number of emitted lights is limited for each of the light emitting elements associated with the plurality of element group regions. The light detection element according to claim 10, wherein the control circuit limits the number of emissions from each of the light-emitting elements associated with the plurality of element group regions based on the reaction state of each of the plurality of element group regions. The light emitting unit has a plurality of light emitting elements,
a light receiving surface of the light receiving unit is divided into a plurality of element group regions,
the plurality of light-emitting elements are associated with the plurality of element group regions based on their positions;
The light detection element according to claim 1 , wherein the light emission is stopped for each element associated with the plurality of element group regions. The light detection element according to claim 12, wherein the control circuit stops the emission of light from each of the elements associated with the plurality of element group regions based on the reaction state of each of the plurality of element group regions. The irradiation light is irradiated a predetermined number of times,
The light detection element according to claim 1 , wherein when the number of times that the plurality of light receiving elements receive light exceeds a predetermined threshold, the control circuit outputs a recovery signal to a main control unit. The irradiation light is irradiated a predetermined number of times,
adding up the number of times that the light is received by each of the plurality of light receiving elements in accordance with the elapsed time from the irradiation timing of the irradiation light;
When the added number exceeds a predetermined threshold value,
The light detection element according to claim 2 , wherein the control circuit limits the number of emitted light beams. The irradiation light is irradiated a predetermined number of times,
adding up the number of times that the light is received by each of the plurality of light receiving elements in accordance with the elapsed time from the irradiation timing of the irradiation light;
When the added number exceeds a predetermined threshold value,
The light detection element according to claim 4 , wherein the control circuit stops the light emission for a predetermined period. The light detection element of claim 1, wherein the frame corresponds to a predetermined control period. A step of receiving light reflected by an object by a plurality of light receiving elements;
a control step of transmitting an information signal including information for controlling a light emission method in frames N and after based on a reaction state of the plurality of light receiving elements in the Nth frame to an irradiation control unit that controls a light emitting unit that emits the irradiation light;
A distance measuring method comprising: a light-emitting unit having a light-emitting portion that emits irradiation light;
an illumination control unit that controls the light emitting unit;
A light detection unit;
A ranging system comprising:
The light detection unit is
a light receiving section having a plurality of light receiving elements that receive light reflected by an object;
a control circuit that transmits an information signal including information for controlling a light emission method in frames N and thereafter based on a reaction state of the plurality of light receiving elements in the Nth frame to an irradiation control unit that controls a light emitting unit that emits the irradiation light;
A ranging system comprising: A main control unit capable of controlling the light detection unit,
The distance measuring system according to claim 19 , wherein the main control unit is in a power saving state when the light receiving unit controls the light emission method.