JP4997604B2 - Personal mobile vehicle driving support device - Google Patents

Description

  The present invention relates to a personal mobile vehicle driving support device that observes a road surface or a floor surface in the vicinity of a personal mobile vehicle, detects an area that cannot be safely moved, and presents it.

  A technology for recognizing obstacles around a vehicle has been developed and applied to vehicle control such as collision avoidance. Many conventional techniques are premised on traveling on a road at a high speed of several tens of kilometers or more. In such a case, an object is to quickly detect an obstacle ahead. In addition, it assumes a space such as a paved road or a parking lot, and the road surface is assumed to be uniform. Sensors such as cameras, stereo cameras, laser range finders, and millimeter wave radars are used. Sensor fusion technology is also used.

In Patent Document 1, an obstacle (dangerous part) detected by a height detection device such as a laser range finder is displayed superimposed on an image taken by a camera.
Patent Document 2 detects a three-dimensional object around a vehicle by fusing information from a stereo camera, millimeter wave radar, and laser radar.
Japanese Patent Application Laid-Open No. H10-228707 converts an image of a camera attached to the periphery of the vehicle, and displays an image as if taken from above the vehicle.
In Patent Document 4, an obstacle is detected at high speed by a stereo camera.

On the other hand, personal mobile vehicles, such as electric wheelchairs and senior cars, run at a low speed of 6 km / h or less. It is more important to prevent the accidental fall of the driver by detecting the dangerous area. In addition, a personal moving vehicle may move anywhere as long as it is a space where a human moves on foot, such as indoors or outdoors. The road surface and floor surface of these spaces are rich in diversity and cannot be expected to be as uniform as a paved road.
JP 2006-160193 A JP 2007-310741 A JP 2007-89081 JP 2001-273488 A C. Harris and M. Stephens, “A combined corner and edge detector”, Proc. 4th Alvey Vision Conference, pp.147-151 (1998) DG Lowe, “Object recognition from local scale-invariant features”, Proc. IEEE International Conference on Computer Vision (ICCV), pp.1150-1157 (1999)

  The present invention is based on a color image feature obtained by observing a region in the vicinity of a vehicle and a three-dimensional shape feature, so that a personal moving vehicle can safely move in an environment where humans indoors and outdoors move on foot. An object of the present invention is to provide a device for estimating the movement safety of the vehicle and presenting it to the driver while performing driving control.

The personal mobile vehicle driving support apparatus of the present invention observes the road surface or floor surface in the vicinity of the personal mobile vehicle, detects and presents a region where it cannot move safely. The personal mobile vehicle driving support device includes a sensor device that observes a road surface or a floor surface in the vicinity of the vehicle and outputs a two-dimensional color image and a three-dimensional image, and a two-dimensional color image and a three-dimensional image output by the sensor device. Are temporarily stored and output to the image processing unit and the three-dimensional data processing unit, and the two-dimensional color image and the three-dimensional image are analyzed to obtain a two-dimensional image feature and a three-dimensional shape feature, respectively. and the image processing unit outputs and three-dimensional data processing unit, and a feature storage unit for registering the 2-dimensional image features and 3D shape features, the 2-dimensional image features and 3D shape features, in advance in the feature storage unit by comparison with the past road history traveled by the vehicle which is registered, to estimate the safety of the road surface around the vehicle, and a movable area estimation unit for outputting a risk map, before In addition to the three-dimensional configuration characteristic, the two-dimensional image features obtained from the two-dimensional color image may be to use the same time, improve the safety estimation accuracy around the vehicle.

  According to the present invention, indoor and outdoor humans can be estimated by estimating the safety of movement in the vicinity area from the color image features and three-dimensional shape characteristics obtained by observing the area in the vicinity of the vehicle and presenting it to the driver. It becomes possible for a personal mobile vehicle to move safely in an environment that moves on foot.

  Hereinafter, the present invention will be described based on examples. FIG. 1 is a system configuration diagram showing a first embodiment of a personal mobile vehicle driving support apparatus. The personal mobile vehicle driving support device of the present invention observes roads and floors in the vicinity of a personal mobile vehicle such as an electric wheelchair or an electric senior car, detects a region where the mobile vehicle cannot be safely moved, and serves as a driver or an automatic driving system. Present. The sensor device 1 includes a plurality of sensors installed around the vehicle, observes a road surface and a floor surface near the vehicle, and outputs a two-dimensional color image and a three-dimensional image.

  FIG. 2 is a diagram showing an image of sensor attachment. As shown in FIG. 2, sensors are arranged around the vehicle, the structure near the wheel is observed, and the observed data is integrated as a risk map and presented to the driver, and the driver steers to the dangerous area. When the vehicle is operated, the driver is warned by sound and vibration. In general vehicles that move at high speeds on road surfaces that can be expected to be uniform, such as paved roads, sensors such as stereo cameras, laser range finders, and millimeter wave radars can be used to detect obstacles in the middle and long distances ahead of the vehicle. Often it is estimated whether the vehicle can move safely. However, in the case of a personal mobile vehicle that moves in various spaces at low speeds, it is more important to detect structures such as steps and grooves close to the wheels in order to prevent the driver from inadvertently or accidentally falling over. .

  An image processing unit 2 and a three-dimensional data processing unit 3 shown in FIG. 1 analyze a color image and a three-dimensional image, respectively, and output a two-dimensional image feature and a three-dimensional shape feature. On a road surface that can be expected to be uniform, such as a paved road, it is often estimated whether or not the vehicle can move safely by simply detecting the three-dimensional shape of the obstacle using a sensor such as a stereo camera or a laser range finder. However, since the space in which a personal mobile vehicle moves is much more diverse than a general paved road, it is necessary to consider various factors.

  For example, grooves and steps are typical dangerous road surfaces and floor surfaces, but these can be detected by measuring a three-dimensional shape. However, even if it is a three-dimensional flat road / floor surface, it may be a soft ground that is difficult to escape with a personal mobile vehicle such as muddy. FIG. 3 is a diagram illustrating an example of a dangerous road surface / floor surface. It is difficult to judge the danger of such a road surface only by measuring a three-dimensional shape.

  The present invention improves the safety estimation accuracy around the vehicle by simultaneously using the two-dimensional image feature obtained from the color image in addition to the three-dimensional shape feature. Two-dimensional image features include color information such as hue and brightness, edge features obtained by differentiating images, features obtained by basic image processing algorithms such as texture features, corner features (Harris & Stephens, 1998), Various features such as SIFT features (Lowe, 1999) can be used. Non-patent documents 1 and 2 can be referred to as references for such two-dimensional image features. For example, a stereo camera system can be used as a sensor that can simultaneously observe a three-dimensional shape and a color image.

  The temporary storage unit 4 shown in FIG. 1 is a memory (image buffer) that temporarily stores data. The latest color image and 3D image observed by the sensor device are accumulated in the temporary storage unit as a time-series image over a plurality of frames together with their attributes, and input to the image processing unit and the 3D data processing unit. The The feature storage unit 5 is a feature database that can register image features and shape features together with their attributes. Here, the attribute is a parameter value related to the feature. For example, when the color information is an image feature, the attribute includes a hue, brightness, a color histogram, and the edge feature attributes include edge strength and edge. There are directions. Attributes of the shape feature include the height of the step, the width / depth of the groove, the height / width of the protrusion, and the like.

  As described above, the space in which the personal moving vehicle moves is far more diverse than a general paved road. FIG. 4 is a diagram for explaining risk determination of an unknown road surface by comparison with a known road surface. As shown in FIG. 4, the safety of the road surface around the vehicle is estimated by comparing the two-dimensional image feature and the three-dimensional shape feature obtained from the above-mentioned sensor with the history of the road surface on which the vehicle has traveled in the past. Known image features / shape features detected from an area that can be safely or cannot be moved are registered in advance in the feature storage unit 5 (FIG. 1). For example, three-dimensional shape features that are obvious to be dangerous for any vehicle, such as deep grooves and sharp protrusions, and two-dimensional image features obtained from the colors, shapes, and characters of default signs drawn on the floor This is the case. Further, since it can be considered that the road surface on which the vehicle has traveled has been confirmed to be safe, the features detected from the area are dynamically registered in one or both of the feature storage unit and the temporary storage unit. Both the previously registered known image feature / shape feature and the image feature / shape feature dynamically registered as the vehicle travels are used for analysis of an unknown road surface. Since it is considered that the road surface on which the personal moving vehicle moves has locality, it can be expected to improve adaptability to an unknown road surface in safety estimation by dynamically registering a small number of features.

  The movable region estimation unit 6 shown in FIG. 1 is obtained by analyzing the latest observation image of the feature registered in the feature storage unit 5 or the feature obtained from the data stored in the temporary storage unit 4. By comparing with the feature, it is estimated whether or not the vehicle can safely move to the image region to which the feature belongs, and is output as a risk map. The data update unit 7 registers the risk map in one or both of the feature storage unit 5 and the temporary storage unit 4. Also, information on the shape feature / image feature of either or both of the feature storage unit and the temporary storage unit is updated.

  Since it is predicted that various uncertain factors are often included in the newly estimated safety estimation of the region, the final decision as to whether or not the estimation result is correct is made by the driver. The system dynamically updates the parameters of the database by indirect interaction through steering performed by the driver. Regardless of the estimation result for the observed area, the safety of the area is confirmed by the driver actually moving the vehicle to the area. If the safety of the area is confirmed, the attribute value (parameter) of the feature storage unit is updated. The movement information of the vehicle is obtained from the steering angle and the travel accumulated distance input from the driving control device. That is, when the driver operates the vehicle ignoring the warning, the error of the attribute value registered in the device is corrected, and the accuracy of risk determination after that is improved. The driver does not need direct interaction with the device and can concentrate on driving operations.

  The image processing program storage unit 8 is a database for registering a plurality of image processing programs based on different algorithms, and an appropriate program for analyzing newly observed road surfaces and floor surfaces is provided to the image processing unit. . There are countless image processing algorithms for analyzing two-dimensional color images, and there is no universal algorithm. Therefore, it is necessary to appropriately select an algorithm according to the purpose, but it is difficult to cope with a fixed image processing algorithm because the space in which the personal moving vehicle moves varies. This problem is addressed by registering image processing programs based on a plurality of different algorithms in the image processing program database and having a mechanism for dynamically applying the image processing program. It is also possible to replace the image processing program according to the locality of the environment in which the vehicle travels. As the image processing program to be registered, the above-described algorithms for detecting various image features can be used.

  The risk map shown in FIG. 1 is presented to a human driver through the display device 9. The acoustic device 10 and the vibration device 11 are devices for issuing a warning when a human driver steers in the direction of an area where it cannot move safely. Information about an area that cannot be moved safely is directly output to the automatic driving device of the vehicle. Information such as the steering angle of the vehicle and the total travel distance is input from the vehicle operation control device. Reference numerals 2 to 8 in the figure are recorded in the storage device 12 as system programs, and are expanded to a computer system or the like when the device is operated.

FIG. 5 is a system configuration diagram showing a second embodiment of the personal mobile vehicle driving support device.
The data update unit 7 is different from the first embodiment in that the risk update map is registered only in the feature storage unit 5. Further, the data update unit 7 updates information on the shape feature / image feature in the feature storage unit 5.

FIG. 6 is a system configuration diagram showing a third embodiment of the personal mobile vehicle driving support apparatus.
The data updating unit 7 is different from the first and second embodiments in that the risk level map is registered only in the temporary storage unit 4. Further, the data update unit 7 updates information on the shape feature / image feature in the temporary storage unit 4.

1 is a system configuration diagram showing a first embodiment of a personal mobile vehicle driving support device. FIG. It is a figure which shows the attachment image of a sensor. It is a figure which shows the example of a dangerous road surface / floor surface. It is a figure explaining danger judgment of an unknown road surface by comparison with a known road surface. It is a system block diagram which shows 2nd Embodiment of a personal mobile vehicle driving assistance device. It is a system block diagram which shows 3rd Embodiment of a personal mobile vehicle driving assistance device.

Claims (6)

In the personal mobile vehicle driving support device that observes the road surface or floor surface in the vicinity of the personal mobile vehicle, detects an area that cannot be safely moved, and presents it,
A sensor device that observes a road surface or a floor surface in the vicinity of the vehicle and outputs a two-dimensional color image and a three-dimensional image;
A temporary storage unit that temporarily accumulates a two-dimensional color image and a three-dimensional image output by the sensor device, and outputs them to an image processing unit and a three-dimensional data processing unit;
By analyzing the two-dimensional color image and three-dimensional images, and the image processing unit and the three-dimensional data processing unit for outputting the 2D image features and the three-dimensional configuration characteristic, respectively,
A feature storage unit for registering the two-dimensional image feature and the three-dimensional shape feature;
By comparing the two-dimensional image feature and the three-dimensional shape feature with the history of the road surface on which the vehicle has traveled in the past registered in the feature storage unit in advance, the safety of the road surface around the vehicle can be estimated and A movable region estimation unit that outputs as a degree map,
In addition to the three-dimensional configuration characteristic, the two-dimensional image features obtained from the two-dimensional color image may be to use the same time, personal mobile vehicle driving support device consists of improving the safety estimation accuracy around the vehicle.   In the feature storage unit, a two-dimensional image feature and a three-dimensional shape feature detected from an area that can be safely moved or cannot be moved are registered in advance. By comparing the 2D image feature and 3D shape feature registered in the feature storage unit with the 2D image feature and 3D shape feature obtained by analyzing the latest observed image, the 2D image feature and The personal mobile vehicle driving support device according to claim 1, wherein it is estimated whether or not the vehicle can safely move to an image region to which the three-dimensional shape feature belongs.   It is considered that the road surface on which the vehicle has traveled has been confirmed to be safe, and the two-dimensional image feature and the three-dimensional shape feature detected from the area are dynamically registered in one or both of the feature storage unit and the temporary storage unit. The personal mobile vehicle driving support device according to claim 1 or 2, wherein the two-dimensional image feature and the three-dimensional shape feature registered in advance are used for analysis of an unknown road surface. Information on the two-dimensional image feature and three-dimensional shape feature of either or both of the feature storage unit and the temporary storage unit, and the risk map is registered in either or both of the feature storage unit and the temporary storage unit personal mobile vehicle driving support device according to any one of claims 1 to 3 comprises a data updating unit for updating the. An image processing program storage unit that registers a plurality of image processing programs based on different algorithms and provides the image processing unit with an appropriate program for analyzing a newly observed road surface / floor surface. personal mobile vehicle driving support device according to any one of 4. The risk map is presented to the driver through a display device, or a warning is issued to the driver through an audio device or a vibration device, or information on an area where the vehicle cannot be safely moved is provided to the automatic driving device of the vehicle. personal mobile vehicle driving support device according to any one of claims 1 to 5 output directly.