JP4899647B2 - Distance measuring program, distance measuring device, distance measuring method - Google Patents

Description

  The present invention relates to a distance measurement program, a distance measurement device, and a distance measurement method for measuring the distance between a moving object and an obstacle.

  A stereo distance measurement method for measuring the distance between a moving vehicle and an obstacle using a monocular imaging unit fixed to the moving vehicle will be described. FIG. 13 is a plan view showing an example of a conventional distance measuring method using stereo vision. Images are taken at different times at two points, and triangulation is performed from the correspondence of the same feature points. Here, when the parallax is x, the focal length of the imaging unit is f, the distance between the imaging points is BL, and the distance from the focal point to the feature point is L, the relationship between these values is expressed by the following equation.

  In addition, L is expressed by the following equation according to Equation 1.

  Further, according to Equation 2, the change ΔL in L accompanying the change in parallax is expressed by the following equation.

Here, f is a value specific to the imaging unit, and here is constant because it is a consideration when a monocular imaging unit is used. Further, since an image is used as an information source for feature point search, a quantization error Δx occurs. Δx is a value that depends on the resolution of the imaging unit and the size of the imaging device, and is constant.

As a related art related to the present invention, a time-series image is captured by a single camera provided in a moving vehicle, a three-dimensional object is detected based on images at different points with parallax, and the moving vehicle is detected. There is a parking assistance device that calculates a distance to a three-dimensional object (see, for example, Patent Document 1). In addition, a mobile object periphery monitoring device that constructs a three-dimensional environment based on images obtained by capturing the periphery of a moving vehicle at at least two points and determines whether or not there is a region larger than a size that can be parked around the moving vehicle. Yes (see, for example, Patent Document 2).
JP 2001-187553 A JP 2004-149977 A

  FIG. 14 is a table showing problems of a conventional distance measuring method using stereo vision. As shown in this table, according to Equation 2, when the distance BL between the two imaging points is short, the parallax x becomes small, so that the ratio of error in the parallax (relative error in parallax) increases. On the other hand, when the distance BL between the two imaging points is long, the parallax x becomes large, and the relative error of the parallax becomes small. However, since the search range for the same feature point between the two images to be used is widened, it tends to fall into a local solution. Furthermore, there is a problem that the calculation cost increases due to the large parallax.

Further, according to Equation 3, it is necessary to increase BL in order to reduce the error | ΔL | of the distance L to be obtained, and to make | ΔL | constant, it is necessary to set an appropriate BL for L. There is.

  Further, in Patent Document 1, since the parallax is calculated from images captured only at two different points by a single camera, as described above, the parallax is calculated when the distance between the two cameras is short. The relative error of distance (parallax) becomes large. On the other hand, when the distance between the two cameras is long, the feature point search range is widened, so that the same feature point search tends to fall into a local solution and the distance (parallax) is not reliable. Further, Patent Document 1 attempts to reduce the search cost by predicting the position of the same feature point in two images using vehicle movement data. However, of course, since the parallax is not known, there is a high possibility that the prediction will be lost, and the reliability is lacking. Further, in Patent Document 2, as in Patent Document 1, a case where a sufficient parallax cannot be obtained is not considered.

  FIG. 15 is a plan view showing an example of a conventional method for measuring the depth of a parking space by stereo vision. This figure shows a method for measuring the depth of the parking space r (the distance Lp from the moving vehicle t to the point p and the distance Lq from the moving vehicle t to the point q) using the imaging unit s fixed to the moving vehicle t. Is shown. Here, a case is considered where the depth is calculated by a conventional method that does not take into account the case where sufficient parallax cannot be obtained as in Patent Document 1 and Patent Document 2.

Here, it is assumed that Lp = 1.5 [m], Lq = 6.5 [m], f = 3.6e−3 [m], and BL = 1.0e−1 [m]. Further, if the resolution of the imaging unit s is 640 × 480 pixels and the imaging element size is 1/2 inch, Δx = 0.8e−3 / 640. According to Equation 3, each error | ΔLp of the depth Lp and Lq of the parking space
| And | ΔLq | are expressed by the following equations.

  In general, a parking space often has an uneven shape as shown in the example of FIG. 15 (a shape surrounded by three sides of the parking space on both sides and the back wall). The distance between the distance feature points is not constant. When the conventional method is used for a parking space with such an uneven shape, the error is small at the distance measurement feature point p in the foreground, but the error is very large at the distance measurement feature point q in the back, and parking assistance is realized. I can't do it.

Both errors | ΔLp | and | ΔLq | can be reduced by increasing the distance between the imaging points. However, in the distance measurement of the point p, since the same feature point search range is widened, there is a problem that the corresponding point calculation tends to fall into a local solution and the reliability of the distance (parallax) is lacking. As described above, when the conventional method is used, it is difficult to calculate the distance between the moving vehicle whose distance is not constant and the feature point for distance measurement with high accuracy.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide a distance measurement program, a distance measurement device, and a distance measurement method that improve the accuracy of measurement of the distance between a moving object and an obstacle. And

  In order to solve the above-described problem, the present invention is a distance measurement program that causes a computer to measure the distance between a moving body and an obstacle, and a plurality of images are continuously acquired by an imaging unit provided in the moving body. An imaging step of capturing an image of a frame, detecting an imaging position and orientation that is the position and orientation of the imaging, and recording the frame and the imaging position and orientation in association with each other; and included in the frame imaged by the imaging step Extracting feature points of the obstacle to make the frame a reference frame and a reference frame and using the feature points as a reference feature point and a reference feature point; and a reference frame determined by the frame determination step A search feature point that is a feature point corresponding to the reference feature point in a search frame that is a frame imaged at a later time A search step for searching; a reference feature point determined by the search step; a search feature point determined by the search step; and an imaging position and orientation recorded by the imaging step; A distance calculating step for calculating a distance between the feature point and a distance calculation step that determines whether or not an error in the distance calculated in the distance calculating step satisfies a predetermined condition. An error determination step for outputting the distance calculated in the distance calculation step as a fixed distance is executed by a computer.

  According to this distance measurement program, the accuracy of measurement can be improved regardless of the distance between the moving object and the obstacle by performing stereo vision using two frames in which the error in the distance satisfies a predetermined condition. .

  Further, in the distance measurement program according to the present invention, when the error of the distance calculated at the end does not satisfy the above condition, the latest search frame is set as a new reference frame and the latest search feature point is newly referenced. A frame re-determination step as a feature point, and a frame captured at a time later than the reference frame determined in the frame re-determination step as a new search frame, and a search feature corresponding to the reference feature point in the search frame A stereo search is performed based on a re-search step for searching for a point, a reference feature point determined by the re-search step, a search feature point determined by the re-search step, and an imaging position and orientation recorded by the imaging step A distance recalculation step for calculating a distance between the moving object and the feature point; and the distance recalculation step. Determining whether or not the error of the distance calculated by the above condition satisfies the condition, and repeating the frame re-determination step, the re-search step, and the distance re-calculation step until the error satisfies the condition, When the error satisfies the condition, the computer is caused to execute an error redetermination step of outputting the distance calculated by the latest distance recalculation step as a confirmed distance.

  According to this distance measurement program, feature points are searched in a predetermined search range regardless of the distance between the moving object and the obstacle, and the error of the distance is repeated until the predetermined condition is satisfied, thereby improving the measurement accuracy. Can be made.

  In the distance measurement program according to the present invention, the condition is that the error in the distance is a predetermined threshold value or less.

  According to this distance measurement program, the distance error can be suppressed to a predetermined threshold value or less regardless of the distance between the moving object and the obstacle.

Moreover, what applied the component of this invention, or arbitrary combinations of a component to a method, an apparatus, a system, a recording medium, a data structure, etc. is also contained in this invention.

  ADVANTAGE OF THE INVENTION According to this invention, the precision of the measurement of the distance of a mobile body and an obstruction can be improved.

  Embodiments of the present invention will be described below with reference to the drawings.

  In the present embodiment, a parking assistance device to which the distance measuring device of the present invention is applied will be described.

  First, the structure of the parking assistance apparatus according to the present embodiment will be described.

  FIG. 1 is a block diagram illustrating an example of the configuration of the parking assistance apparatus according to the first embodiment. This parking assistance device is realized by a computer provided in a moving vehicle (moving body) or a device connected thereto. In addition, the parking assist device captures a vehicle state detection unit 21 that detects the position of the moving vehicle and the movement of the moving vehicle, captures the periphery of the moving vehicle, acquires frames (images) continuous in time, and captures a single eye. Unit 11 (camera), a frame 12 captured by the imaging unit 11, and a memory 12 that stores an imaging point (imaging position) that is a position detected by the vehicle state detection unit 21 at the time when the frame was captured, A feature point extraction unit 13 that extracts a feature point from the frame; a distance calculation unit 14 that calculates the position and orientation of the moving vehicle and an obstacle around the target parking space based on the feature point, and the width and depth of the target parking space; Position and orientation of the moving vehicle calculated by the distance calculation unit 14 (frame determination unit, search unit, distance calculation unit, error determination unit) and obstacles around the parking space, width and depth of the target parking space A three-dimensional model generation unit 15 (obstacle shape generation unit) that generates a three-dimensional model of the target parking space, a vehicle outer dimension storage unit 16 that stores an outer dimension of the moving vehicle in advance, and a three-dimensional model and determination unit A determination unit 17 (contact determination unit) that determines whether or not parking is possible, and a display unit 18 that transmits a determination result by the determination unit 17 to the driver.

  Here, the vehicle state detection unit 21 detects the position and orientation of the moving vehicle using, for example, a vehicle speed pulse, a steering angle, or a gyro sensor.

  Next, the operation of the parking assistance apparatus according to the present embodiment will be described.

  FIG. 2 is a flowchart showing an example of the operation of the parking assistance apparatus according to the first embodiment. First, the parking determination apparatus is activated by the driver of the moving vehicle (S111). Thereafter, the driver performs a parking operation (such as garage parking or parallel parking) of the moving vehicle. FIG. 3 is a plan view showing an example of the garage parking operation of the moving vehicle according to the first embodiment. FIG. 4 is a plan view illustrating an example of parallel parking operation of the moving vehicle according to the first embodiment. 3 and 4, the moving vehicle is backed and parked at the parking target position after reaching the position shown in the figure. At that time, the driver of the moving vehicle performs the parking operation with reference to the output result of the present apparatus described later.

  Next, the moving vehicle is in a state as shown in FIGS. 3 and 4, and the vehicle state detection unit 21 determines whether or not the vehicle speed of the moving vehicle is within a predetermined range indicating a low speed (S112). If the moving vehicle is not slow (S112, NO), the process returns to S112. On the other hand, when the moving vehicle becomes low speed (S112, YES), the vehicle state detection unit 21 determines that the moving vehicle passes through the target parking space at low speed (preparation action for parking), and the imaging unit 11 sets the time series. A plurality of consecutive frames are imaged, the captured frames are stored in the memory 12, and the vehicle state detection unit 21 corresponds to an imaging point (imaging position and orientation) indicating a position and orientation at the time of imaging the frame. Then, it is stored in the memory 12 (S113) (imaging step). Next, the feature point extraction unit 13 extracts feature points in each frame stored in the memory 12, and stores the coordinates of the feature points in the memory 12 (S121).

  Here, instead of the imaging unit 11 storing all the frames in the memory 12, the plane coordinates of the feature points extracted by the feature point extraction unit 13 may be stored in the memory 12. Thereby, the memory 12 does not need to store all the frames imaged by the imaging unit 11, and it is only necessary to store the correspondence relationship of the same feature points, so that the capacity of the memory 12 can be reduced.

  Next, the distance calculation unit 14 determines the position and orientation of the moving vehicle and obstacles around the parking space, the width and depth of the target parking space, based on the position and orientation of the moving vehicle at the time of imaging and the feature points of the plurality of frames. Is calculated (S122). Next, the three-dimensional model generation unit 15 generates a three-dimensional model of the target parking space from the position and orientation of the moving vehicle calculated by the distance calculation unit 14 and the obstacles around the parking space and the width and depth of the target parking space. (S123) (obstacle shape generation step). Next, it is determined whether or not parking is possible in the determination unit 17 from this result and the external shape of the moving vehicle stored in the vehicle external dimension storage unit 16 in advance (S131) (contact determination step). Next, the display unit 18 displays whether or not parking is possible according to the determination result, and transmits it to the driver (S132).

  In the present embodiment, the display unit 18 (shape display unit) displays the determination result. However, the 3D model generated by the 3D model generation unit 15 may be displayed (shape display step). Further, the display unit 18 may display the outer shape of the moving vehicle stored in the vehicle outer dimension storage unit 16 on the three-dimensional model in accordance with the position and orientation of the vehicle by the vehicle state detection unit 21.

  Next, details of the operation of the distance calculation unit 14 will be described.

  FIG. 5 is a flowchart illustrating an example of the operation of the distance calculation unit 14 according to the first embodiment. First, the distance calculation unit 14 sets the feature point number n = 1 (S10), and based on the imaging point corresponding to the feature point in the memory 12, the imaging point where the feature point Pn first appears (captured) is tk. The appearance start position frame (base frame, reference frame) that is a frame imaged at the imaging point is set as the first image F0 (S11) (frame determination step). In addition, frames subsequent to the frame F0 in time series are F1, F2, F3,.

  Here, the specific example of the parking operation for description of operation | movement of the distance calculation part 14 is shown. FIG. 6 is a plan view showing an example of the parking operation according to the first embodiment. This figure shows a garage parking operation in which a moving vehicle parks in an uneven target parking space. Here, imaging points, which are points where images are taken, are defined as t1, t2, t3, t4,... T8, t9. Further, the feature points extracted by the feature point extraction unit 13 are P1, P2, P3, and P4. FIG. 7 is a graph showing an example of the relationship between the feature points and the imaging points according to the first embodiment. In the example of FIG. 6, imaging points at which the feature point P1 is imaged are t1 to t3, imaging points at which the feature point P2 is imaged are t3 to t9, and imaging points at which the feature point P3 is imaged are t2 to t8. The imaging points where P4 is imaged are t8 to t9.

  Next, the distance calculation unit 14 sets the frame number m = 1 (S12), and is the same in the second image Fm (search frame) with respect to the feature points (standard feature point, reference feature point) in the first image F0. A feature point (search feature point) is searched (search step, re-search step), and between the same feature points (reference feature point and search feature) in the first image F0 (reference frame) and the second image Fm (search frame). A distance Lm (parallax Xm) between the points is calculated (distance calculation step, distance recalculation step) (S13). Here, with respect to the feature point P1, from the images at the imaging points t1 and t2, from the image at the imaging points t3 and t4 for the feature point P2, and from the images at the imaging points t2 and t3 for the feature point P3. For the feature point P4, the distance Lm (parallax Xm) is calculated from the images at the imaging point t8 and the imaging point t9.

Thereafter, the distance calculation unit 14 determines whether or not the distance error | ΔL | obtained from Equation 3 is equal to or smaller than a preset threshold value (S14) (error determination step, error re-determination step). When | ΔL | is equal to or smaller than the threshold value (S14, YES), the distance calculation unit 14
When the process proceeds to step S16 and | ΔL | is larger than the threshold (S14, NO), the second
As a new reference frame, a feature point searched in the reference frame as a reference feature point, m = m + 1, the second image Fm as the next search frame, and the same feature point as the reference feature point (search feature) Point) is searched (S15), and the flow from step S13 is repeated until | ΔL |

FIG. 8 is a diagram illustrating an example of operations of processes S13 to S15 according to the first embodiment. First, the distance calculation unit 14 has two images (F0 and F1) having the shortest distance between the imaging points (the smallest parallax) among a plurality of consecutive (N) images captured at different imaging points. The distance L1 (parallax) between the same feature points at is calculated. Next, the distance calculation unit 14 determines that two images (F0 and F0) that are next separated from each other are separated so that the error ratio in the parallax decreases if | ΔL | in Equation 3 is greater than a predetermined threshold. The process of calculating the parallax between the same feature points (between the reference feature points and the search feature points) from F2) is repeated a plurality of times (m times) until | ΔL | becomes a predetermined threshold value or less. As a result, the distance calculation unit 14 determines the first image (F0) and the second image (Fm) to be used for distance measurement, and uses these images to determine the distance between the feature point and the vehicle ( Lm). In addition, since the parking space often has an uneven shape, the first image and the second image are selected for each feature point to be measured.

  Here, the difference between the same feature point search of the prior art and the same feature point search of the present embodiment will be described.

  FIG. 9 is a diagram illustrating an example of a conventional identical feature point search. This figure shows a case where a feature point identical to the feature point in the first image is searched for in the second image. The shaded part in the second image is a search range for the same feature point, and a square search range with a side length of 3x is set around the feature point in the first image. As described above, in the conventional method, if a large parallax is to be obtained, the search range needs to be widened. Therefore, there are many feature points that may be erroneously recognized, and the same feature point search tends to fall into a local solution.

  FIG. 10 is a diagram illustrating an example of the same feature point search according to the first embodiment. In this figure, in order from the top, when searching for the same feature point (search feature point) as the feature point (reference feature point) in the frame F0 (reference frame) in the next frame F1 (search frame), the frame F1 (reference) When searching for the same feature point (search feature point) as the feature point (reference feature point) in the frame) in the next frame F2 (search frame), it is the same as the feature point (reference feature point) in the frame F2 (reference frame) This is a case where the feature point (search feature point) is searched in the next frame F3 (search frame). The shaded portion in each frame is a search range for the same feature point, and a square search range having a side length of x is set around the feature point in the previous frame. As a result, 3 × parallax (distance between the reference feature point and the latest search feature point) can be obtained.

  That is, in order to obtain the same 3x parallax, in FIG. 9, which is a conventional search for the same feature point, a search range having a side of 3x is set, whereas in FIG. 10 which is the same feature point search of the present embodiment. Then, since the search range of x with a small side is set and divided into three times, a smaller search range is required, which solves the conventional problem and increases the reliability of the calculated distance (parallax). Become.

  FIG. 11 is a diagram illustrating an example of parallax with the same feature points as a plurality of images captured during the parking operation according to the first embodiment. This figure is selected to obtain the respective parallaxes of the feature points P1, P2, P3, and P4 from the frames imaged at the imaging points t1, t2, t3, t4, t8, and t9 in the example of FIG. Represents a set of frames. The distance calculation unit 14 can select an optimal frame set for obtaining an appropriate parallax for each feature point by the operations of the processes S13 to S15 described above.

  In the example of FIG. 6, the distance calculation unit 14 proceeds to processing S16 because the parallax is sufficiently large (distance accuracy is high) when the images of the imaging point t1 and the imaging point t2 are used for the distance measurement of the feature point P1. Further, the distance calculation unit 14 proceeds to the process S15 because the distance accuracy is low if the images of the imaging point t3 and the imaging point t4 are used for the distance measurement of the feature point P2, and the imaging point t3 and the imaging point where sufficient parallax can be obtained. The image at t9 is used. Further, the distance calculation unit 14 proceeds to step S15 because the distance accuracy is low when the images of the imaging point t2 and the imaging point t3 are used for the distance measurement of the feature point P3, and the imaging point t2 and the imaging point where sufficient parallax can be obtained. The image at t8 is used. Further, the distance calculation unit 14 proceeds to the process S16 because the parallax is sufficiently large (distance accuracy is high) in the images of the imaging point t8 and the imaging point t9 in the distance measurement of the feature point P4.

  In process S16, the distance calculation unit 14 outputs the distance Lm calculated as described above as the distance between the moving vehicle and the feature point Pn. In step S17, it is determined whether distances (parallaxes) have been calculated for all feature points in the imaging point tk (S17). If all are calculated (S17, YES), this flow ends. On the other hand, if not all have been calculated (S17, NO), the distance calculation unit 14 sets n = n + 1 (S18), and repeats the flow from step S11.

  According to the present embodiment, two images having a parallax sufficient for distance calculation are measured from a plurality of images continuously captured in time series by the imaging unit 11 provided in the moving vehicle. Since each distance feature point is selected, the distance between the vehicle and the distance measurement feature point that is not constant can be measured with high accuracy, and the reliability of the calculated distance is improved. be able to.

Embodiment 2. FIG.
In the present embodiment, a parking assistance device that controls the frame rate of the imaging unit 11 will be described.

  First, the structure of the parking assistance apparatus according to the present embodiment will be described.

  FIG. 12 is a block diagram illustrating an example of the configuration of the parking assistance apparatus according to the second embodiment. In this figure, the same reference numerals as those in FIG. 1 denote the same or corresponding parts as those in FIG. 1, and the description thereof is omitted here. Compared with FIG. 1, this figure newly includes an imaging instruction unit 22. The imaging instruction unit 22 sets the frame rate (imaging timing) of the imaging unit 11 according to the vehicle speed of the moving vehicle detected by the vehicle state detection unit 21 and instructs the imaging unit 11. The imaging unit 11 images the periphery of the moving vehicle in accordance with an instruction from the imaging instruction unit 22.

  The imaging instruction unit 22 controls the frame rate so that the distance between the imaging points (for example, BL) is maintained at a certain value or more regardless of the vehicle speed or the like. Alternatively, unnecessary images are thinned out with respect to the determined frame rate among a plurality of consecutive captured images.

  Here, the association between the vehicle speed of the moving vehicle and the frame rate of the imaging unit 11 by the imaging instruction unit 22 will be described.

  In FIG. 13, if the minimum value of the distance L that needs to be calculated with high accuracy is Lmin, and the parallax x with that accuracy is xmin, the minimum value BLmin of the distance BL between the imaging points is expressed by the following equation (1). expressed.

  Therefore, an image having a BL shorter than BLmin (an image taken at an imaging point at a distance shorter than BLmin from a reference frame) is unnecessary. Further, assuming that the vehicle speed is v (m / s) and the frame rate of the imaging unit 11 is R (frame / s), the minimum distance between imaging points (between one frame) that can be taken under this condition is as follows: It is expressed by a formula.

  Therefore, the distance between imaging points separated by n frames (n is an integer of 1 or more) is expressed by the following equation.

Furthermore, the following formulas are obtained from Formulas 5 and 7.

  Therefore, it is only necessary to control n so that the minimum n satisfying Equation 8 is obtained. That is, the frame rate is set to R / n (frame / s) so as to satisfy Formula 8, or n = of a plurality of time-series continuous images captured at the frame rate R (frame / s). It suffices to thin out 1, 2,..., N-1 frames so that the distance between imaging points used for parallax calculation is v / R × n.

  According to the present embodiment, the distance between the imaging points can be maintained at a certain value or more by imaging by the imaging unit 11 according to the frame rate instructed by the imaging instruction unit 22, or the imaging instruction unit 22 An unnecessary image can be thinned out with respect to a required accuracy among a plurality of consecutive images captured in response to the instruction.

  Moreover, the distance measuring device according to the present embodiment can be easily applied to the in-vehicle information processing device, and the performance of the in-vehicle information processing device can be further improved. Here, the in-vehicle information processing apparatus may include, for example, a car navigation system, an ECU (Electric Control Unit), and the like.

  Furthermore, a program for executing the above-described steps in a computer constituting the distance measuring device can be provided as a distance measuring program. By storing the above-described program in a computer-readable recording medium, the computer constituting the distance measuring apparatus can be executed. Here, examples of the recording medium readable by the computer include an internal storage device such as a ROM and a RAM, a portable storage such as a CD-ROM, a flexible disk, a DVD disk, a magneto-optical disk, and an IC card. It includes a medium, a database holding a computer program, another computer and its database, and a transmission medium on a line.

(Supplementary note 1) A distance measurement program for causing a computer to measure the distance between a moving object and an obstacle,
The imaging unit provided in the moving body continuously captures a plurality of frames, detects an imaging position and orientation that is the position and orientation at which the imaging was performed, and records the frames and the imaging position and orientation in association with each other. Imaging step;
Extracting a feature point of the obstacle included in the frame imaged in the imaging step to make the frame a reference frame and a reference frame, and a frame determination step using the feature point as a reference feature point and a reference feature point;
A search step for searching for a search feature point that is a feature point corresponding to the reference feature point in a search frame that is a frame imaged at a time later than the reference frame determined by the frame determination step;
Stereo vision is executed based on the reference feature point determined by the search step, the search feature point determined by the search step, and the imaging position and orientation recorded by the imaging step, and between the moving object and the feature point A distance calculating step for calculating the distance;
It is determined whether or not the error in the distance calculated in the distance calculation step satisfies a predetermined condition. If the error satisfies the condition, the distance calculated in the distance calculation step is output as a fixed distance. A distance measurement program that causes a computer to execute an error determination step.
(Appendix 2) In the distance measurement program described in Appendix 1,
Further, when the error of the distance calculated at the end does not satisfy the condition, a frame re-determination step in which the latest search frame is a new reference frame and the latest search feature point is a new reference feature point;
Re-search step of searching for a search feature point corresponding to the reference feature point in the search frame, using a frame imaged at a time later than the reference frame determined in the frame re-determination step as a new search frame;
Stereo vision is executed based on the reference feature point determined by the re-search step, the search feature point determined by the re-search step, and the imaging position and orientation recorded by the imaging step, and the moving object and the feature point A distance recalculation step for calculating a distance between;
It is determined whether or not an error in the distance calculated in the distance recalculation step satisfies the condition, and the frame re-determination step, the re-search step, and the distance re-calculation step until the error satisfies the condition. When the error satisfies the above condition, an error re-determination step for outputting the distance calculated by the latest distance re-calculation step as a fixed distance is executed by the computer. .
(Appendix 3) In the distance measurement program described in Appendix 1 or Appendix 2,
The condition is that the distance error is a predetermined threshold value or less.
(Appendix 4) In the distance measurement program described in Appendix 3,
The error of the distance is the distance between the imaging points that is the distance between the position where the reference frame used for calculating the distance is imaged and the position where the search frame used for calculating the distance is imaged, A distance measurement program calculated using a quantization error of parallax in stereo vision and a focal length of the imaging unit.
(Supplementary note 5) In the distance measurement program according to any one of supplementary notes 1 to 4,
Furthermore, the distance measurement program which makes a computer perform the obstruction shape generation step which produces | generates the shape information of the said obstruction based on the said fixed distance acquired about several feature points.
(Appendix 6) In the distance measurement program described in Appendix 5,
Further, based on the shape information of the obstacle generated by the obstacle shape generation step, the external shape information of the mobile body recorded in advance, and the position and orientation of the mobile body detected from the mobile body, A distance measurement program for causing a computer to execute a shape display step for displaying a positional relationship between the obstacle and the moving body.
(Appendix 7) In the distance measurement program described in Appendix 5,
Further, it is determined whether or not the moving body contacts the obstacle based on the obstacle shape information generated by the obstacle shape generation step and the outer shape information of the moving body recorded in advance. A distance measurement program for causing a computer to execute a contact determination step of notifying a driver of the moving body of a determination result.
(Supplementary note 8) In the distance measurement program according to any one of supplementary notes 1 to 7,
The distance measurement program characterized in that the imaging step maintains an interval between positions where the frames are captured based on a change in the position and orientation of the moving body.
(Supplementary note 9) In the distance measurement program according to any one of supplementary notes 1 to 7,
The distance measurement program characterized in that the imaging step thins out the frames so that an interval between positions where the frames are imaged is equal to or greater than a predetermined value based on a change in position and posture of the moving body.
(Supplementary Note 10) A distance measuring device for measuring a distance between a moving object and an obstacle,
An imaging unit that is provided on the moving body and continuously captures images of a plurality of frames, detects an imaging position and orientation that is the position and orientation of the imaging, and records the frames and the imaging position and orientation in association with each other; ,
A frame determination unit that extracts feature points of the obstacle included in the frame imaged by the imaging unit and sets the frame as a reference frame and a reference frame, and uses the feature point as a reference feature point and a reference feature point;
A search unit that searches for a search feature point that is a feature point corresponding to the reference feature point in a search frame that is a frame imaged at a time later than the reference frame determined by the frame determination unit;
Stereo vision is executed based on the reference feature point determined by the search unit, the search feature point determined by the search unit, and the imaging position and orientation recorded by the imaging unit, and between the moving object and the feature point A distance calculation unit for calculating the distance;
It is determined whether or not an error in the distance calculated by the distance calculation unit satisfies a predetermined condition. If the error satisfies the condition, the distance calculated by the distance calculation unit is output as a fixed distance. A distance measuring device comprising an error determination unit.
(Supplementary note 11) In the distance measuring device according to supplementary note 10,
The frame determination unit, when the error of the distance calculated last does not satisfy the condition, frame re-determination using the latest search frame as a new reference frame and the latest search feature point as a new reference feature point Process,
The search unit sets, as a new search frame, a frame imaged at a time later than the reference frame determined by the frame redetermination process, and searches for a search feature point corresponding to the reference feature point in the search frame. Perform a search process,
The distance calculation unit performs stereo vision based on the reference feature point determined by the re-search process, the search feature point determined by the re-search process, and the imaging position and orientation recorded by the imaging unit, Perform a distance recalculation process to calculate the distance between the moving object and the feature point,
The error determination unit determines whether or not an error in the distance calculated by the distance recalculation process satisfies the condition, and the frame redetermination process and the re-search process until the error satisfies the condition. And the distance recalculation process are repeated, and if the error satisfies the condition, the distance calculated by the latest distance recalculation process is output as a fixed distance.
(Supplementary Note 12) In the distance measuring device according to Supplementary Note 10 or Supplementary Note 11,
The condition is that the distance error is not more than a predetermined threshold value.
(Supplementary note 13) In the distance measuring device according to supplementary note 12,
The error of the distance is the distance between the imaging points that is the distance between the position where the reference frame used for calculating the distance is imaged and the position where the search frame used for calculating the distance is imaged, A distance measuring device calculated using a quantization error of parallax in stereo vision and a focal length of the imaging unit.
(Supplementary Note 14) In the distance measuring device according to any one of Supplementary Notes 10 to 13,
The distance measuring device further includes an obstacle shape generation unit that generates shape information of the obstacle based on the determined distance obtained for a plurality of feature points.
(Supplementary Note 15) In the distance measuring device according to Supplementary Note 14,
Furthermore, based on the shape information of the obstacle generated by the obstacle shape generation unit, the external shape information of the mobile body recorded in advance, and the position and orientation of the mobile body detected from the mobile body, A distance measuring apparatus comprising a shape display unit for displaying a positional relationship between the obstacle and the moving body.
(Supplementary note 16) In the distance measuring device according to supplementary note 14,
Further, it is determined whether or not the moving body contacts the obstacle based on the obstacle shape information generated by the obstacle shape generating unit and the outer shape information of the moving body recorded in advance. A distance measurement device comprising a contact determination unit that notifies a determination result to a driver of the moving body.
(Supplementary Note 17) In the distance measuring device according to any one of Supplementary Notes 10 to 16,
The distance measuring apparatus according to claim 1, wherein the imaging unit maintains an interval between positions where the frame is imaged at a predetermined value or more based on a change in position and posture of the moving body.
(Supplementary note 18) In the distance measuring device according to any one of supplementary notes 10 to 16,
The distance measuring apparatus according to claim 1, wherein the imaging unit thins out the frames based on a change in the position and orientation of the moving body so that an interval between positions where the frames are captured is equal to or greater than a predetermined value.
(Supplementary note 19) A distance measuring method for measuring a distance between a moving object and an obstacle,
The imaging unit provided in the moving body continuously captures a plurality of frames, detects an imaging position and orientation that is the position and orientation at which the imaging was performed, and records the frames and the imaging position and orientation in association with each other. Imaging step;
Extracting a feature point of the obstacle included in the frame imaged in the imaging step to make the frame a reference frame and a reference frame, and a frame determination step using the feature point as a reference feature point and a reference feature point;
A search step for searching for a search feature point that is a feature point corresponding to the reference feature point in a search frame that is a frame imaged at a time later than the reference frame determined by the frame determination step;
Stereo vision is executed based on the reference feature point determined by the search step, the search feature point determined by the search step, and the imaging position and orientation recorded by the imaging step, and between the moving object and the feature point A distance calculating step for calculating the distance;
It is determined whether or not the error in the distance calculated in the distance calculation step satisfies a predetermined condition. If the error satisfies the condition, the distance calculated in the distance calculation step is output as a fixed distance. A distance measurement method that executes an error determination step.
(Supplementary note 20) In the distance measurement method according to supplementary note 19,
Further, when the error of the distance calculated at the end does not satisfy the condition, a frame re-determination step in which the latest search frame is a new reference frame and the latest search feature point is a new reference feature point;
Re-search step of searching for a search feature point corresponding to the reference feature point in the search frame, using a frame imaged at a time later than the reference frame determined in the frame re-determination step as a new search frame;
Stereo vision is executed based on the reference feature point determined by the re-search step, the search feature point determined by the re-search step, and the imaging position and orientation recorded by the imaging step, and the moving object and the feature point A distance recalculation step for calculating a distance between;
It is determined whether or not an error in the distance calculated in the distance recalculation step satisfies the condition, and the frame re-determination step, the re-search step, and the distance re-calculation step until the error satisfies the condition. When the error satisfies the above condition, an error re-determination step is executed to output the distance calculated by the latest distance re-calculation step as a fixed distance.

It is a block diagram which shows an example of a structure of the parking assistance apparatus which concerns on Embodiment 1. FIG. 4 is a flowchart showing an example of the operation of the parking assistance device according to the first embodiment. FIG. 6 is a plan view showing an example of a garage parking operation of the moving vehicle according to the first embodiment. 6 is a plan view showing an example of parallel parking operation of the moving vehicle according to Embodiment 1. FIG. 6 is a plan view illustrating an example of a parking operation according to Embodiment 1. FIG. 4 is a graph illustrating an example of a relationship between feature points and imaging points according to the first embodiment. 3 is a flowchart illustrating an example of an operation of a distance calculation unit 14 according to the first embodiment. It is a figure which shows an example of operation | movement of process S13-S15 which concerns on Embodiment 1. FIG. It is a figure which shows an example of the conventional same feature point search. 6 is a diagram showing an example of the same feature point search according to Embodiment 1. FIG. 6 is a plan view showing an example of parallax of the same feature points as a plurality of images taken during the parking operation according to Embodiment 1. FIG. It is a block diagram which shows an example of a structure of the parking assistance apparatus which concerns on Embodiment 2. FIG. It is a top view which shows an example of the distance measuring method by the conventional stereo vision. It is a table | surface which shows the problem in the conventional stereo vision. It is a top view which shows an example of the depth measuring method of the parking space by the conventional stereo vision.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Image pick-up part, 12 Memory, 13 Feature point extraction part, 14 Distance calculation part, 15 3D model production | generation part, 16 External dimension memory | storage part, 17 Determination part, 18 Display part, 21 Vehicle state detection part, 22 Imaging instruction | indication part

Claims (5)

A distance measurement program for causing a computer to measure a distance between a moving object and an obstacle,
The imaging unit provided in the moving body continuously captures a plurality of frames, detects an imaging position and orientation that is the position and orientation at which the imaging was performed, and records the frames and the imaging position and orientation in association with each other. Imaging step;
Extracting a feature point of the obstacle included in the frame imaged in the imaging step to make the frame a reference frame and a reference frame, and a frame determination step using the feature point as a reference feature point and a reference feature point;
A search step for searching for a search feature point that is a feature point corresponding to the reference feature point in a search frame that is a frame imaged at a time later than the reference frame determined by the frame determination step;
Stereo vision is executed based on the reference feature point determined by the frame determination step, the search feature point determined by the search step, and the imaging position and orientation recorded by the imaging step, and between the moving object and the feature point A distance calculating step for calculating the distance of
It is determined whether or not the error in the distance calculated in the distance calculation step satisfies a predetermined condition. If the error satisfies the condition, the distance calculated in the distance calculation step is output as a fixed distance. An error determination step ;
If the last calculated distance error does not satisfy the above condition, a frame re-determination step using the latest search frame as a new reference frame and the latest search feature point as a new reference feature point;
Re-search step of searching for a search feature point corresponding to the reference feature point in the search frame, using a frame imaged at a time later than the reference frame determined in the frame re-determination step as a new search frame;
Stereo vision is executed based on the reference feature point determined in the frame determination step, the search feature point determined in the re-search step, and the imaging position and orientation recorded in the imaging step, and the moving object and the feature point A distance recalculation step for calculating a distance between;
It is determined whether or not an error in the distance calculated in the distance recalculation step satisfies the condition, and the frame re-determination step, the re-search step, and the distance re-calculation step until the error satisfies the condition. When the error satisfies the above condition, a distance measurement program that causes the computer to execute an error redetermination step that outputs the distance calculated by the latest distance recalculation step as a determined distance. The distance measurement program according to claim 1 ,
The condition is that the distance error is a predetermined threshold value or less. In the distance measurement program according to claim 1 or 2,
The re-search step sets a search range based on the calculated parallax with the search feature point searched in the immediately preceding frame as a center. A distance measuring device for measuring a distance between a moving object and an obstacle,
An imaging unit that is provided on the moving body and continuously captures images of a plurality of frames, detects an imaging position and orientation that is the position and orientation of the imaging, and records the frames and the imaging position and orientation in association with each other; ,
A frame determination unit that extracts feature points of the obstacle included in the frame imaged by the imaging unit and sets the frame as a reference frame and a reference frame, and uses the feature point as a reference feature point and a reference feature point;
A search unit that searches for a search feature point that is a feature point corresponding to the reference feature point in a search frame that is a frame imaged at a time later than the reference frame determined by the frame determination unit;
Stereo vision is executed based on the reference feature point determined by the frame determination unit, the search feature point determined by the search unit, and the imaging position and orientation recorded by the imaging unit, and between the moving object and the feature point A distance calculation unit for calculating the distance of
It is determined whether or not an error in the distance calculated by the distance calculation unit satisfies a predetermined condition. If the error satisfies the condition, the distance calculated by the distance calculation unit is output as a fixed distance. An error determination unit ,
The frame determination unit, when the error of the distance calculated last does not satisfy the condition, frame re-determination using the latest search frame as a new reference frame and the latest search feature point as a new reference feature point Process,
The search unit sets, as a new search frame, a frame imaged at a time later than the reference frame determined by the frame redetermination process, and searches for a search feature point corresponding to the reference feature point in the search frame. Perform a search process,
The distance calculation unit performs stereo vision based on the reference feature point determined by the frame determination unit, the search feature point determined by the re-search process, and the imaging position and orientation recorded by the imaging unit, Perform a distance recalculation process to calculate the distance between the moving object and the feature point,
The error determination unit determines whether or not an error in the distance calculated by the distance recalculation process satisfies the condition, and the frame redetermination process and the re-search process until the error satisfies the condition. and repeating the said distance recalculation processing, if said error is the condition is satisfied, distance measuring device which outputs a distance to confirm the distance calculated by the latest of the distance recalculation processing. A distance measurement method for measuring a distance between a moving object and an obstacle,
The imaging unit provided in the moving body continuously captures a plurality of frames, detects an imaging position and orientation that is the position and orientation at which the imaging was performed, and records the frames and the imaging position and orientation in association with each other. Imaging step;
Extracting a feature point of the obstacle included in the frame imaged in the imaging step to make the frame a reference frame and a reference frame, and a frame determination step using the feature point as a reference feature point and a reference feature point;
A search step for searching for a search feature point that is a feature point corresponding to the reference feature point in a search frame that is a frame imaged at a time later than the reference frame determined by the frame determination step;
Stereo vision is executed based on the reference feature point determined by the frame determination step, the search feature point determined by the search step, and the imaging position and orientation recorded by the imaging step, and between the moving object and the feature point A distance calculating step for calculating the distance of
It is determined whether or not the error in the distance calculated in the distance calculation step satisfies a predetermined condition. If the error satisfies the condition, the distance calculated in the distance calculation step is output as a fixed distance. An error determination step ;
If the last calculated distance error does not satisfy the above condition, a frame re-determination step using the latest search frame as a new reference frame and the latest search feature point as a new reference feature point;
Re-search step of searching for a search feature point corresponding to the reference feature point in the search frame, using a frame imaged at a time later than the reference frame determined in the frame re-determination step as a new search frame;
Stereo vision is executed based on the reference feature point determined in the frame determination step, the search feature point determined in the re-search step, and the imaging position and orientation recorded in the imaging step, and the moving object and the feature point A distance recalculation step for calculating a distance between;
It is determined whether or not an error in the distance calculated in the distance recalculation step satisfies the condition, and the frame re-determination step, the re-search step, and the distance re-calculation step until the error satisfies the condition. When the error satisfies the above condition, an error re-determination step that outputs the distance calculated by the latest distance re-calculation step as a fixed distance is executed.

Images