JP2024137371A - Electronic device, distance measurement method, and distance measurement program - Google Patents

Abstract

[Problem] To provide an electronic device capable of improving the accuracy of distance measurement from a moving body to an object. [Solution] The electronic device 10 according to the present disclosure is provided with an imaging unit 11 that is disposed on a moving body 1 and includes a first imaging unit 111 and a second imaging unit 112 that face in different directions, and a control unit 13 that performs triangulation in a common field of view V of the two imaging units 11 and measures the distance to an object S by motion stereo from the movement distance of the imaging unit 11 obtained based on the triangulation. [Selected Figure] Figure 1

Description

This disclosure relates to an electronic device, a distance measurement method, and a distance measurement program.

Conventionally, distance measurement technology is known that measures the distance from a moving object such as a vehicle to an object using an imaging unit such as a camera arranged on the moving object. For example, Patent Document 1 discloses a stereo camera system and a distance measurement method that reduce the processing load for distance measurement by the stereo camera.

JP 2020-190925 A

However, conventional technology leaves room for improvement in the accuracy of measuring the distance from a moving body to a target.

The present disclosure aims to provide an electronic device, a distance measurement method, and a distance measurement program that can improve the accuracy of measuring the distance from a moving body to a target.

According to a first aspect of the present invention, there is provided an electronic device comprising:
An imaging unit that is disposed on the moving body and includes a first imaging unit and a second imaging unit that face in different directions;
a control unit that performs triangulation in a common field of view of the two imaging units and measures a distance to an object by motion stereo from a moving distance of the imaging units obtained based on the triangulation;
Equipped with.

The distance measuring method according to the second aspect comprises:
A step of performing triangulation in a common field of view of an imaging unit including a first imaging unit and a second imaging unit that are disposed on a moving body and face in different directions;
measuring a distance to an object by motion stereo from a moving distance of the imaging unit obtained based on the triangulation;
Includes.

The third aspect of the distance measurement program is
For electronic devices,
A step of performing triangulation in a common field of view of an imaging unit including a first imaging unit and a second imaging unit that are disposed on a moving body and face in different directions;
measuring a distance to an object by motion stereo from a moving distance of the imaging unit obtained based on the triangulation;
The method executes an operation including:

The present disclosure makes it possible to provide an electronic device, a distance measurement method, and a distance measurement program that can improve the accuracy of measuring distances from a moving body to a target.

1 is a schematic diagram showing distance measurement performed by a moving object having an electronic device according to an embodiment of the present disclosure; FIG. 1 is a block diagram showing an example of a configuration of an electronic device according to an embodiment of the present disclosure. FIG. 3 is a first diagram for explaining an example of processing by the electronic device of FIG. 2 . FIG. 3 is a second diagram for explaining an example of processing by the electronic device of FIG. 2 . FIG. 3 is a third diagram for explaining an example of processing by the electronic device of FIG. 2 . FIG. 4 is a fourth diagram for explaining an example of processing by the electronic device of FIG. 2 . FIG. 5 is a diagram for explaining an example of processing by the electronic device of FIG. 2 . 3 is a flowchart illustrating an example of an operation of the electronic device of FIG. 2 .

Below, one embodiment of the present disclosure will be mainly described with reference to the attached drawings. The following description also applies to a ranging method and a ranging program to which the present disclosure is applied.

FIG. 1 is a schematic diagram showing distance measurement by a mobile object 1 having an electronic device 10 according to an embodiment of the present disclosure. FIG. 2 is a block diagram showing an example of the configuration of an electronic device 10 according to an embodiment of the present disclosure. With reference to FIGS. 1 and 2, the configuration of the mobile object 1 and the electronic device 10 possessed by the mobile object 1, and an example of distance measurement by the electronic device 10 possessed by the mobile object 1 will be mainly described.

The electronic device 10 according to an embodiment of the present disclosure is mounted on, for example, a moving body 1. In the present disclosure, the "moving body 1" includes vehicles, ships, aircraft, drones, and the like. The "vehicles" include automobiles, industrial vehicles, railroad vehicles, lifestyle vehicles, and fixed-wing aircraft that run on runways. The "automobiles" include passenger cars, trucks, buses, motorcycles, trolley buses, and the like. The "industrial vehicles" include industrial vehicles for agriculture and construction, and the like. The industrial vehicles include forklifts and golf carts, and the like. The industrial vehicles for agriculture include tractors, cultivators, transplanters, binders, combines, and lawnmowers, and the like. The industrial vehicles for construction include bulldozers, scrapers, excavators, cranes, dump trucks, and road rollers, and the like. The vehicles include those that run by automatic driving, those that run based on the driving by a driver, and those that run by human power. The classification of vehicles is not limited to the above examples. For example, the automobile may include an industrial vehicle that can run on a road. The same vehicle may be included in multiple classifications. "Marines" includes passenger ships, watercraft, boats, and tankers. "Aircraft" includes fixed-wing and rotary-wing aircraft. "Drones" includes drones for flying and underwater mobility, etc.

2, the electronic device 10 has a first imaging section 111, a second imaging section 112, a third imaging section 113, and a fourth imaging section 114, a storage section 12, and a control section 13. Hereinafter, when the first imaging section 111, the second imaging section 112, the third imaging section 113, and the fourth imaging section 114 are not to be distinguished from one another, they will be referred to as "imaging section 11." The first imaging section 111, the second imaging section 112, the third imaging section 113, and the fourth imaging section 114 are disposed in the moving body 1.

The first imaging unit 111 faces the moving body 1 in a first direction and captures a scene in the first direction. The first imaging unit 111 outputs the captured scene to the control unit 13 as a first image. In this disclosure, the "first image" includes, for example, at least one still image and a video including a plurality of still images successively as frames. The first imaging unit 111 includes, for example, a first camera. The first camera includes a monocular camera having a fisheye lens. Without being limited thereto, the first camera may include a monocular camera having a viewing angle of 180° to nearly 360° using an ultra-wide-angle lens other than a fisheye lens. The first camera may include a monocular camera having a wide-angle lens. The first camera may include any other monocular camera having a viewing angle capable of forming a common field of view V, which will be described later with reference to FIG. 1, between adjacent cameras. In addition, the first camera is not limited to a monocular camera and may include a compound eye camera.

The second imaging unit 112 faces the moving body 1 in a second direction and captures a scene in the second direction. The second imaging unit 112 outputs the captured scene to the control unit 13 as a second image. In this disclosure, the "second image" includes, for example, at least one still image and a video including a plurality of still images successively as frames. The second imaging unit 112 includes, for example, a second camera. The second camera includes a monocular camera having a fisheye lens. Without being limited thereto, the second camera may include a monocular camera having a viewing angle of 180° to nearly 360° using an ultra-wide-angle lens other than a fisheye lens. The second camera may include a monocular camera having a wide-angle lens. The second camera may include any other monocular camera having a viewing angle capable of forming a common field of view V between adjacent cameras. In addition, the second camera is not limited to a monocular camera and may include a compound eye camera.

The third imaging unit 113 faces the moving body 1 in the third direction and captures a scene in the third direction. The third imaging unit 113 outputs the captured scene to the control unit 13 as a third image. In this disclosure, the "third image" includes, for example, at least one still image and a video including a plurality of still images successively as frames. The third imaging unit 113 includes, for example, a third camera. The third camera includes a monocular camera having a fisheye lens. Without being limited thereto, the third camera may include a monocular camera having a viewing angle of 180° to nearly 360° using an ultra-wide-angle lens other than a fisheye lens. The third camera may include a monocular camera having a wide-angle lens. The third camera may include any other monocular camera having a viewing angle capable of forming a common field of view V between adjacent cameras. In addition, the third camera is not limited to a monocular camera and may include a compound eye camera.

The fourth imaging unit 114 faces the moving body 1 in the fourth direction and captures a scene in the fourth direction. The fourth imaging unit 114 outputs the captured scene to the control unit 13 as a fourth image. In this disclosure, the "fourth image" includes, for example, at least one still image and a video including a plurality of still images successively as frames. The fourth imaging unit 114 includes, for example, a fourth camera. The fourth camera includes a monocular camera having a fisheye lens. Without being limited thereto, the fourth camera may include a monocular camera having a viewing angle of 180° to nearly 360° using an ultra-wide-angle lens other than a fisheye lens. The fourth camera may include a monocular camera having a wide-angle lens. The fourth camera may include any other monocular camera having a viewing angle capable of forming a common field of view V between adjacent cameras. In addition, the fourth camera is not limited to a monocular camera and may include a compound eye camera.

The storage unit 12 includes storage devices such as a hard disk drive (HDD), a solid state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), and a random access memory (RAM). The storage unit 12 stores information necessary to realize the operation of the electronic device 10. The storage unit 12 stores information obtained by the operation of the electronic device 10. For example, the storage unit 12 stores system programs, application programs, and various data obtained by any means such as communication.

The memory unit 12 may function as a main memory device, an auxiliary memory device, or a cache memory. The memory unit 12 is not limited to being built into the electronic device 10, but may be an external memory device connected via a digital input/output port such as a Universal Serial Bus (USB).

The control unit 13 includes one or more processors. In this disclosure, a "processor" refers to, but is not limited to, a general-purpose processor or a dedicated processor specialized for a particular process. The control unit 13 is communicatively connected to each component constituting the electronic device 10, and controls the operation of the entire electronic device 10. For example, the control unit 13 may constitute a part of an ECU (Electronic Control Unit) possessed by the mobile object 1 as a vehicle.

As shown in FIG. 1, the four imaging units 11 including the first imaging unit 111, the second imaging unit 112, the third imaging unit 113, and the fourth imaging unit 114 are disposed on the moving body 1 and face in different directions. For example, when the moving body 1 includes a vehicle, the first imaging unit 111, the second imaging unit 112, the third imaging unit 113, and the fourth imaging unit 114 are disposed so as to face in four directions, namely, the front, rear, left, and right, of the vehicle.

As an example, the first direction in which the first imaging unit 111 faces from the moving body 1 corresponds to the forward direction. The second direction in which the second imaging unit 112 faces from the moving body 1 corresponds to the rightward direction. The third direction in which the third imaging unit 113 faces from the moving body 1 corresponds to the backward direction. The fourth direction in which the fourth imaging unit 114 faces from the moving body 1 corresponds to the leftward direction. The four cameras included in each of the four imaging units 11 constitute an all-around camera in the moving body 1.

The first camera of the first imaging unit 111, the second camera of the second imaging unit 112, the third camera of the third imaging unit 113, and the fourth camera of the fourth imaging unit 114 each have a viewing angle close to 180°. Due to the wide viewing angle of each camera, a common field of view V is formed by a pair of adjacent cameras. For example, a first common field of view V1 is formed in the pair of the first imaging unit 111 and the second imaging unit 112. For example, a second common field of view V2 is formed in the pair of the second imaging unit 112 and the third imaging unit 113. For example, a third common field of view V3 is formed in the pair of the third imaging unit 113 and the fourth imaging unit 114. For example, a fourth common field of view V4 is formed in the pair of the fourth imaging unit 114 and the first imaging unit 111.

The electronic device 10 possessed by the moving body 1 detects an object S that exists in front of the moving body 1 in the moving direction, and an object S that exists in another direction, and supports the movement of the moving body 1. In this disclosure, "object S" includes objects, people such as pedestrians, and other moving bodies moving around the moving body 1. The electronic device 10 notifies an operator who is controlling the movement of the moving body 1 of information related to the detected object S. For example, the electronic device 10 measures the distance between the detected object S and the moving body 1, and predicts the risk of collision between the object S and the moving body 1 according to the distance. The electronic device 10 notifies the operator of the risk of collision and executes a warning process.

The electronic device 10 performs triangulation processing using a set of imaging units 11 that have a common field of view V, thereby performing motion stereo ranging processing using each camera with high accuracy. An example of processing performed by the electronic device 10 is described in more detail below.

Figure 3 is a first diagram for explaining an example of processing by the electronic device 10 of Figure 2. The control unit 13 of the electronic device 10 generates a parallax image from a pair of frame images having a chronological relationship, for example, using each camera constituting a panoramic camera in the moving body 1. Below, the parallax image generation process will be explained using the second camera of the second imaging unit 112 as an example, but a similar explanation applies to the other imaging units 11.

(a) in FIG. 3 is a schematic diagram showing an example of the appearance of a frame image at time t. (b) in FIG. 3 is a schematic diagram showing an example of the appearance of a frame image at time t-1. The two schematic diagrams shown in FIG. 3 show, as an example, the appearance of a second image captured by the second camera of the second imaging unit 112, which is disposed on the right side of the moving body 1 and faces in the right direction.

The moving object 1 is moving forward at a predetermined speed. Therefore, the four imaging units 11 arranged on the moving object 1 also change their positions over time in accordance with the movement of the moving object 1. For example, the second imaging unit 112 also changes its position as time passes from time t-1 to time t. As a result, the frame image at time t-1 and the frame image at time t are captured from different imaging positions for the second imaging unit 112. Parallax occurs between these frame images.

The control unit 13 of the electronic device 10 generates a parallax image of two frame images based on the frame image at time t-1 and the frame image at time t. The control unit 13 generates the parallax image in motion stereo using any method, such as a stereo matching method or a plane sweep method.

The control unit 13 calculates the depth distance Z from the parallax D included in the generated parallax image based on the following formula 1.
(Equation 1)
In formula 1, B represents the base line length, i.e., the moving distance of the imaging unit 11. B represents the length of the line connecting the position of the camera at time t and the position of the camera at time t-1. f represents the focal length of the camera included in the imaging unit 11.

In this disclosure, "parallax D" is, for example, the amount of movement on an image of a specific corresponding point of target S that occurs between a frame image at time t and a frame image at time t-1, converted into a physical distance in real space. Parallax D is converted from the pixel difference for a specific corresponding point between the above two frame images into a physical distance by multiplying it by a coefficient based on information such as the pixel size of each pixel. "Depth distance Z" is, for example, the distance from a camera included in the imaging unit 11 to a specific corresponding point of target S along the optical axis of the camera.

In motion stereo, the control unit 13 needs to obtain the baseline length B, focal length f, and parallax D as information to calculate the depth distance Z according to the above formula 1, but it is not easy to calculate the baseline length B as an absolute distance from an image. Therefore, with motion stereo alone, the value of the baseline length B is unknown, and it is not easy to accurately calculate the depth distance Z.

Therefore, the control unit 13 performs triangulation in the common field of view V of at least one of the four imaging units 11, and calculates the baseline length B as the movement distance of the imaging units 11 based on the triangulation. The control unit 13 measures the distance to the target S by motion stereo from the movement distance of the imaging units 11 obtained based on the triangulation. The following mainly describes the calculation process of the baseline length B using triangulation.

Figure 4 is a second diagram for explaining an example of processing by the electronic device 10 of Figure 2. Below, the calculation process of the baseline length B based on triangulation will be explained using an example of a pair of the first camera of the first imaging unit 111 and the second camera of the second imaging unit 112, but a similar explanation also applies to other pairs of imaging units 11.

(a) in FIG. 4 is a schematic diagram showing an example of a first image captured by the first camera of the first imaging unit 111 facing forward on the moving body 1. (b) in FIG. 4 is a schematic diagram showing an example of a second image captured by the second camera of the second imaging unit 112 facing right on the moving body 1. The two images shown as schematic diagrams in FIG. 4 were captured simultaneously at a predetermined timing while the moving body 1 was moving.

As described above, in the omnidirectional cameras mounted on the moving body 1, there exists an area where the fields of view of adjacent cameras overlap, which is called a common field of view V. When executing the triangulation process, the control unit 13 calculates a corresponding point _Pi in the common field of view V of the two images shown in Fig. 4. For example, the control unit 13 calculates a corresponding point Pi from the first common field of view V1 between a first image captured by the first camera of the first imaging unit 111 and a second image captured by the second camera of the second imaging unit 112. The control unit 13 executes the triangulation process based on _the calculated corresponding point _Pi .

Figure 5 is a third diagram for explaining an example of processing by the electronic device 10 of Figure 2. In the triangulation processing executed by the control unit 13, the following calculation is performed. For example, in Figure 5, the distance L between the first camera of the first imaging unit 111 and the second camera of the second imaging unit 112 is expressed by the following formula 2.

(Equation 2)
In Equation 2, Z' represents the depth distance when the two cameras are used as references. Z' represents the length of a perpendicular line dropped from the corresponding point P _i to a line of length L connecting the first camera and the second camera. α represents the angle between the line of the light vector r _α extending from the first camera to the corresponding point P _i and the line of length L connecting the first camera and the second camera. β represents the angle between the line of the light vector r _β extending from the second camera to the corresponding point P _i and the line of length L connecting the first camera and the second camera.

By modifying Equation 2, the depth distance Z′ is obtained as shown in Equation 3 below.
(Equation 3)

On the other hand, the control unit 13 calculates the depth distance Z from the second camera of the second imaging unit 112 to the corresponding point P _i using the following formula based on Z′ calculated by triangulation.
In Equation 4, |r _β | represents the length of the light vector r _β . In Equation 5, r _βz represents the component of the light vector r _β in the depth distance Z direction. The control unit 13 calculates the length of the light vector r _β based on Equation 4 using Z' calculated using Equation 3. The control unit 13 calculates the depth distance Z using Equation 5 based on the length of the light vector r _β calculated using Equation 4 and the known angle of the light vector r _β from the depth distance Z direction.

The control unit 13 executes a similar calculation process for all corresponding points P _i (i=1, 2, ..., n) and calculates a depth distance Z _i for each corresponding point P _i . The control unit 13 also executes a similar calculation process for the first camera of the first imaging unit 111 adjacent to the second camera of the second imaging unit 112. The control unit 13 calculates a depth distance Z _i for each corresponding point P _i for both the first camera and the second camera.

Each corresponding point P _i has information on a disparity D _{i that} can be obtained from a disparity image and a depth distance Z _i obtained by triangulation. Therefore, a base line length B _i is calculated from these two parameters.

The control unit 13 calculates the base line length B _i as the movement distance of the imaging unit 11 based on the parallax D _{i in the parallax image including the corresponding point P i in} the common field of view V generated by the above-mentioned motion stereo and the depth distance Z _i from the imaging unit 11 to the corresponding point P i obtained by triangulation. For example, the control unit 13 calculates the base line length B _i by the following formula ₆ based on the parallax D _i in the parallax image generated by motion stereo using the second camera of the second imaging unit 112 and the depth distance Z _i from the second camera to the corresponding point P _i obtained by triangulation.

The baseline length B _i calculated by Equation 6 is the moving distance of the imaging unit 11 accompanying the movement of the moving object 1, and therefore ideally, it should be the same value for all corresponding points P _i . However, the values of the baseline length B _i vary due to the influence of various erroneous matching, and outliers are generated as noise. In the present disclosure, "erroneous matching" includes a case where the parallax D _i cannot be obtained correctly due to missing image information and noise, etc., when a parallax image is generated from two frame images using a specific imaging unit 11, and a case where the corresponding points P _i do not match between the two cameras in triangulation.

Fig. 6 is a fourth diagram for explaining an example of processing by the electronic device 10 of Fig. 2. In order to exclude the above-mentioned outlier value M2 of the base line length B _i , the control unit 13 finally calculates the moving distance of the imaging unit 11 used for measuring the distance of the target S by statistical processing based on the most frequent value M1 of the base line length B _i calculated for the multiple corresponding points P _i .

6, the control unit 13 creates a histogram of the number of corresponding points P _i for each of a plurality of baseline lengths B _i obtained for a given camera of the imaging unit 11. The control unit 13 excludes an outlier M2 and calculates the average value of the remaining baseline lengths B _i around the mode M1.

The control unit 13 uses the calculated average value as the final baseline length B, i.e., the movement distance of the imaging unit 11, for the motion stereo distance measurement process. The control unit 13 calculates the depth distance Z based on the average value and the parallax D in the parallax image according to Equation 1. This enables the control unit 13 to generate an accurate depth image over the entire field of view of the camera of a given imaging unit 11, and to measure the distance to all objects S present in the depth image.

7 is a fifth diagram for explaining an example of processing by the electronic device 10 of FIG. 2. In the above description, the baseline length B is calculated based on statistical processing in one camera, but this is not limiting. The control unit 13 may finally calculate the moving distance of the imaging unit 11 used for measuring the distance to the target S by statistical processing based on the most frequent value M1 of the baseline lengths B _i calculated in the multiple common fields of view V in the multiple sets of imaging units 11.

The base length B _i calculated by Equation 6 is the moving distance of the imaging unit 11 accompanying the movement of the moving object 1, so it is ideal that the base length B i is the same value for all cameras. However, for example, in a nighttime scene, the front of the moving object 1 is illuminated by lighting, so it is possible to calculate the corresponding point P _i in the first common field of view V1 and the fourth common field of view V4, but it may not be easy to calculate the corresponding point P _i in the other common fields of view V because they are dark. This makes it more susceptible to the influence of erroneous matching, etc., and makes it difficult to calculate the moving distance of the imaging unit 11. As a result, it is also not easy to measure the distance to the target S by motion stereo. Even in scenes other than nighttime, it is assumed that it is not easy to obtain a sufficient number of corresponding points P _i in all four cameras constituting the omnidirectional camera.

Therefore, assuming the above-mentioned cases, the control unit 13 may integrate and calculate the baseline lengths B _i corresponding to the corresponding points P _i calculated by each camera, thereby integrating information on the movement distance of the entire imaging unit 11 in the moving object 1. For example, as shown in (b) of FIG. 7, the control unit 13 creates a histogram of the number of corresponding points P _i for each of the multiple baseline lengths B _i obtained for the cameras of all the imaging units 11. The control unit 13 excludes the outlier M2 and calculates the average value of the remaining baseline lengths B _i around the mode M1. The mode M1 shown in (b) of FIG. 7 appears statistically more clearly than the mode M1 of the histogram for the multiple baseline lengths B _i obtained only for the second camera of the second imaging unit 112 shown in (a) of FIG.

The control unit 13 uses the calculated average value as the final baseline length B, i.e., the movement distance of the imaging unit 11, in the motion stereo distance measurement process. As described above, the control unit 13 may perform triangulation in the common field of view V of multiple sets of imaging units 11 out of the four imaging units 11, and measure the distance to the target S by motion stereo from the movement distance of the imaging unit 11 obtained based on the triangulation.

Figure 8 is a flowchart illustrating an example of the operation of the electronic device 10 of Figure 2. With reference to Figure 8, an example of the processing of the distance measurement method executed by the electronic device 10 will be described.

In step S100, the control unit 13 of the electronic device 10 captures an image using the imaging unit 11. The control unit 13 stores the image captured using the imaging unit 11 as information in the storage unit 12.

In step S101, the control unit 13 of the electronic device 10 generates a parallax image between the frame image at time t-1 and the frame image at time t based on the image captured in step S100.

In step S102, the control unit 13 of the electronic device 10 performs triangulation in the common field of view V of at least one set of the four imaging units 11 that face in different directions.

In step S103, the control unit 13 of the electronic device 10 calculates the depth distance Z _i from the imaging unit 11 to the corresponding point P _i based on the triangulation performed in step S102.

In step S104, the control unit 13 of the electronic device 10 calculates the movement distance of the imaging unit 11 as a baseline length B _i based on the parallax D _i in the parallax image generated in step S101 and the depth distance Z _i obtained in step S103 by the triangulation in step S102.

In step S105, the control unit 13 of the electronic device 10 measures the distance to the target S using motion stereo based on the movement distance of the imaging unit 11 obtained in step S104.

According to the electronic device 10 according to the embodiment described above, it is possible to improve the accuracy of distance measurement from the moving body 1 to the target S. The electronic device 10 performs triangulation in each common field of view V of at least one set of imaging units 11, and measures the distance to the target S by motion stereo from the obtained movement distance of the imaging units 11. As a result, unlike distance measurement using a conventional monocular camera, the electronic device 10 can accurately measure the distance to the target S even on uneven road surfaces other than flat ground, such as dirt. The electronic device 10 can achieve high-precision distance measurement that is not dependent on the road surface condition. By performing triangulation in addition to motion stereo, the electronic device 10 can accurately calculate the baseline length B required for calculating the depth distance Z in motion stereo as the movement distance of the imaging units 11. As a result, the electronic device 10 can accurately calculate the depth distance Z in motion stereo.

The electronic device 10 calculates the movement distance of the imaging unit 11 based on the parallax D _i in the parallax image and the depth distance Z _i obtained by triangulation, thereby enabling accurate calculation of the base line length B required for calculation of the depth distance Z in motion stereo. Therefore, the electronic device 10 can accurately calculate the depth distance Z in motion stereo.

The electronic device 10 calculates the movement distance of the imaging unit 11 used to measure the distance of the target S by statistical processing based on the most frequent value M1 of the movement distance of the imaging unit 11 calculated for multiple corresponding points P _i , thereby making it possible to increase the number of samples related to the base line length B _i . Therefore, the electronic device 10 can more easily distinguish the outlier M2 generated as noise due to erroneous matching. The electronic device 10 calculates the most frequent value M1 of the base line length B _i , and can more accurately calculate the movement distance of the imaging unit 11 used to measure the distance of the target S. As a result, the electronic device 10 can more accurately calculate the depth distance Z in motion stereo.

The electronic device 10 is disposed on the moving object 1 and further includes a third imaging unit 113 that faces a different direction from the first imaging unit 111 and the second imaging unit 112, thereby enabling distance measurement of the target S over a wider range around the moving object 1 compared to a case in which two imaging units 11 are included. The electronic device 10 can also increase the number of combinations of imaging units 11 that form a common field of view V. In other words, the electronic device 10 can also increase the number of combinations of imaging units 11 that perform triangulation in the common field of view V.

The electronic device 10 performs triangulation in the common field of view V of each of the multiple sets of the three imaging units 11, and measures the distance to the target S by motion stereo from the moving distance of the imaging unit 11 obtained based on the triangulation. This allows the electronic device 10 to further improve the accuracy of measuring the distance from the moving body 1 to the target S. Unlike distance measurement using a conventional monocular camera, the electronic device 10 can accurately measure the distance to the target S even on uneven road surfaces other than flat ground, such as dirt. The electronic device 10 can achieve highly accurate distance measurement that is not dependent on the road surface condition. By performing triangulation using multiple sets of the imaging units 11 in addition to motion stereo, the electronic device 10 can accurately calculate the baseline length B required to calculate the depth distance Z in motion stereo as the moving distance of the imaging unit 11 from multiple angles. As a result, the electronic device 10 can more accurately calculate the depth distance Z in motion stereo.

The electronic device 10 calculates the moving distance of the imaging unit 11 used for measuring the distance of the target S by statistical processing based on the most frequent value M1 of the moving distance of the imaging unit 11 calculated in each of the multiple common fields of view V in the multiple sets of imaging units 11. This allows the electronic device 10 to further increase the number of samples related to the baseline length B _i . Therefore, the electronic device 10 can more easily distinguish the outlier M2 generated as noise due to erroneous matching. The electronic device 10 calculates the most frequent value M1 of the baseline length B _i , and can more accurately calculate the moving distance of the imaging unit 11 used for measuring the distance of the target S. As a result, the electronic device 10 can more accurately calculate the depth distance Z in motion stereo. For example, even if it is difficult to calculate the baseline length B _i because a corresponding point P _i cannot be obtained by a specific camera in a dark scene and against a wall, the electronic device 10 can also complement the baseline length B _i from other cameras.

The electronic device 10 is capable of realizing highly accurate distance measurement all around the vehicle by configuring an all-around sensing system with the first imaging unit 111, the second imaging unit 112, and the third imaging unit 113, each of which is arranged to face one of the front, rear, left and right directions of the vehicle. The electronic device 10 exerts such an effect more prominently by arranging the four imaging units 11 to face the four directions of the front, rear, left and right directions of the vehicle. The electronic device 10 is capable of realizing highly accurate distance measurement all around the vehicle without blind spots. The electronic device 10 is capable of realizing highly accurate distance measurement only with a general-purpose vehicle-mounted camera without requiring an expensive sensor such as LiDAR (Light Detection And Ranging). The electronic device 10 is also capable of realizing highly accurate distance measurement by following a conventional general all-around sensing system.

Although the present disclosure has been described based on the drawings and embodiments, it should be noted that a person skilled in the art would be able to make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each configuration or step can be rearranged so as not to cause logical inconsistencies, and multiple configurations or steps can be combined into one or divided.

For example, the shape, size, arrangement, orientation, and number of each of the above-mentioned components are not limited to the above description and the illustrations in the drawings. The shape, size, arrangement, orientation, and number of each of the components may be configured arbitrarily as long as the function can be realized.

For example, it is also possible to cause a general-purpose terminal device such as a smartphone or a computer to function as part of the electronic device 10 according to the embodiment described above. Specifically, a program describing the processing contents for realizing each function of the electronic device 10 according to the embodiment is stored in the memory of the terminal device, and the program is read and executed by the processor of the terminal device. Therefore, the present disclosure can also be realized as a distance measurement program that can be executed by a processor.

Alternatively, the present disclosure may also be realized as a non-transitory computer-readable medium storing a ranging program executable by one or more processors to cause an electronic device 10 according to an embodiment to execute each function. It should be understood that these are also included within the scope of the present disclosure.

In the above embodiment, the electronic device 10 is described as having four imaging units 11, including a fourth imaging unit 114 that is arranged on the moving object 1 and faces in a different direction from the three imaging units 11, but this is not limited to the above. The electronic device 10 may have three imaging units 11, including a first imaging unit 111, a second imaging unit 112, and a third imaging unit 113, that are arranged on the moving object 1 and face in different directions. In this case, the control unit 13 of the electronic device 10 may perform triangulation in the common field of view V of at least one set of the imaging units 11 out of the three imaging units 11, and measure the distance to the target S by motion stereo from the movement distance of the imaging units 11 obtained based on the triangulation.

The electronic device 10 is not limited to a configuration having three imaging units 11 or four imaging units 11, but may have two imaging units 11, or may have five or more imaging units 11.

In the above embodiment, the electronic device 10 calculates the movement distance of the imaging unit 11 based on the parallax D _i in the parallax image including the corresponding point P _i in the common field of view V and the depth distance Z _i from the imaging unit 11 to the corresponding point P _i obtained by triangulation, but the present invention is not limited to this. The electronic device 10 may calculate the base length B required for calculating the depth distance Z in motion stereo, i.e., the movement distance of the imaging unit 11, by using any other method.

In the above embodiment, the electronic device 10 calculates the moving distance of the image capturing unit 11 used for measuring the distance to the target S by statistical processing based on the most frequent value _M1 of the moving distance of the image capturing unit 11 calculated for the multiple corresponding points Pi, but the present invention is not limited to this. The electronic device 10 does not need to execute such statistical processing.

In the above embodiment, the electronic device 10 calculates the moving distance of the imaging unit 11 used to measure the distance to the target S by statistical processing based on the most frequent value M1 of the moving distance of the imaging unit 11 calculated for each of the multiple common fields of view V in the multiple sets of imaging units 11, but this is not limited to the above. The electronic device 10 does not have to perform such statistical processing.

In the above embodiment, the electronic device 10 has been described as having the first imaging unit 111, the second imaging unit 112, the third imaging unit 113, and the fourth imaging unit 114 arranged to face four directions, namely, the front, rear, left, and right, of the vehicle, but is not limited to this. The four imaging units 11 do not need to be arranged so as to be able to measure distances all around the vehicle, and may be arranged so as to be able to measure distances only in a portion of the entire periphery of the vehicle.

Some of the embodiments of the present disclosure will be described below as examples. However, it should be noted that the embodiments of the present disclosure are not limited to these examples.
[Appendix 1]
An imaging unit that is disposed on the moving body and includes a first imaging unit and a second imaging unit that face in different directions;
a control unit that performs triangulation in a common field of view of the two imaging units and measures a distance to an object by motion stereo from a moving distance of the imaging units obtained based on the triangulation;
Equipped with
Electronic devices.
[Appendix 2]
2. The electronic device according to claim 1,
the control unit calculates the movement distance based on a parallax in a parallax image including the corresponding point in the common field of view generated by the motion stereo and a depth distance from the imaging unit to the corresponding point obtained by the triangulation.
Electronic devices.
[Appendix 3]
3. The electronic device according to claim 2,
the control unit calculates the movement distance used for measuring the distance to the object by statistical processing based on a mode value of the movement distances calculated for the plurality of corresponding points.
Electronic devices.
[Appendix 4]
An electronic device according to any one of claims 1 to 3,
The imaging unit,
The imaging device further includes a third imaging unit that is disposed on the moving body and faces in a direction different from the first imaging unit and the second imaging unit.
Electronic devices.
[Appendix 5]
The electronic device according to claim 4,
The control unit performs triangulation in a common field of view of each of the plurality of sets of the imaging units among the three imaging units, and measures the distance to the object by motion stereo from the movement distance obtained based on the triangulation.
Electronic devices.
[Appendix 6]
6. The electronic device according to claim 5,
the control unit calculates the moving distance used for measuring the distance to the object by statistical processing based on a mode of the moving distances calculated in a plurality of common fields of view in a plurality of sets of the imaging units.
Electronic devices.
[Appendix 7]
7. An electronic device according to any one of claims 4 to 6,
the moving object includes a vehicle,
The first imaging unit, the second imaging unit, and the third imaging unit are each arranged to face any one of the front, rear, left, and right directions of the vehicle.
Electronic devices.
[Appendix 8]
A step of performing triangulation in a common field of view of an imaging unit including a first imaging unit and a second imaging unit that are disposed on a moving body and face in different directions;
measuring a distance to an object by motion stereo from a moving distance of the imaging unit obtained based on the triangulation;
Including,
Distance measurement method.
[Appendix 9]
For electronic devices,
A step of performing triangulation in a common field of view of an imaging unit including a first imaging unit and a second imaging unit that are disposed on a moving body and face in different directions;
measuring a distance to an object by motion stereo from a moving distance of the imaging unit obtained based on the triangulation;
performing an operation including
Ranging program.

REFERENCE SIGNS LIST 1 Mobile object 10 Electronic device 11 Imaging section 111 First imaging section 112 Second imaging section 113 Third imaging section 114 Fourth imaging section 12 Memory section 13 Control section B, _Bi Baseline length D, _Di Parallax f Focal length M1 Mode M2 Outlier P _i Corresponding point S Object V Common field of view V1 First common field of view V2 Second common field of view V3 Third common field of view V4 Fourth common field of view Z, Z _i , Z' Depth distance

Claims (9)

An imaging unit that is disposed on the moving body and includes a first imaging unit and a second imaging unit that face in different directions;
a control unit that performs triangulation in a common field of view of the two imaging units and measures a distance to an object by motion stereo from a moving distance of the imaging units obtained based on the triangulation;
Equipped with
Electronic devices. 2. The electronic device according to claim 1,
the control unit calculates the movement distance based on a parallax in a parallax image including the corresponding point in the common field of view generated by the motion stereo and a depth distance from the imaging unit to the corresponding point obtained by the triangulation.
Electronic devices. 3. The electronic device according to claim 2,
the control unit calculates the movement distance used for measuring the distance to the object by statistical processing based on a mode value of the movement distances calculated for the plurality of corresponding points.
Electronic devices. 4. The electronic device according to claim 1,
The imaging unit,
The imaging device further includes a third imaging unit that is disposed on the moving body and faces in a direction different from the first imaging unit and the second imaging unit.
Electronic devices. 5. The electronic device according to claim 4,
The control unit performs triangulation in a common field of view of each of the plurality of sets of the imaging units among the three imaging units, and measures the distance to the object by motion stereo from the movement distance obtained based on the triangulation.
Electronic devices. 6. The electronic device according to claim 5,
the control unit calculates the moving distance used for measuring the distance to the object by statistical processing based on a mode of the moving distances calculated in a plurality of common fields of view in a plurality of sets of the imaging units.
Electronic devices. 5. The electronic device according to claim 4,
the moving object includes a vehicle,
The first imaging unit, the second imaging unit, and the third imaging unit are each arranged to face any one of the front, rear, left, and right directions of the vehicle.
Electronic devices. A step of performing triangulation in a common field of view of an imaging unit including a first imaging unit and a second imaging unit that are disposed on a moving body and face in different directions;
measuring a distance to an object by motion stereo from a moving distance of the imaging unit obtained based on the triangulation;
Including,
Distance measurement method. For electronic devices,
A step of performing triangulation in a common field of view of an imaging unit including a first imaging unit and a second imaging unit that are disposed on a moving body and face in different directions;
measuring a distance to an object by motion stereo from a moving distance of the imaging unit obtained based on the triangulation;
performing an operation including
Ranging program.

Images