JP6613999B2 - Driving support system - Google Patents

Description

  The present invention relates to a technology for performing driving support by analyzing an image obtained by imaging the periphery of a vehicle.

  ADAS that performs driving support such as FCW and PCW by analyzing an image captured around the vehicle is known. ADAS is also referred to as “Advanced Driver Assistance System” and is an advanced driving assistance system. FCW is also written as “Forward Collision Warning” and is a front vehicle approach warning. PCW is also written as “Pedestrian Collision Warning” and is a passerby warning.

  In Patent Document 1, ADAS wirelessly transmits an image of the surroundings of a vehicle to IVIS, and in IVIS, the received image is analyzed to perform misdirection detection, rainfall detection, wiper control, headlight control, and other operations. Techniques for assisting are disclosed. IVIS is also written as “In-Vehicle Infotainment System” and is an in-vehicle infotainment system.

International Publication No. 2014/077908

  However, there were the following problems. That is, the quick response processing that requires quick response, such as FCW and PCW, is generally performed by ADAS. However, when the responsive processing is performed by ADAS, if there are restrictions on resources allocated to image analysis, high-accuracy image analysis is not performed, and the reliability of the target recognition result by image analysis may be lowered. It was. In addition, when the responsive processing is performed by other devices such as IVIS, it is necessary to constantly transmit and receive image data and image analysis results between ADAS and other devices, which increases the communication burden. was there.

  The present invention has been made in view of these problems, and when performing driving assistance using a target detection result that is a result of detecting a target from a captured image obtained by imaging the periphery of a vehicle, the communication burden of the system It aims at providing the technique which improves the reliability of a target detection result, suppressing this.

  The driving support system (1) of the present invention includes an in-vehicle device (10) and an external device (20). The in-vehicle device is a device mounted on the vehicle (100). The device outside the vehicle is a device provided outside the vehicle. The in-vehicle device and the external device can communicate wirelessly.

The in-vehicle device includes an acquisition unit, a detection unit, a first determination unit, a first generation unit, an execution unit, a second generation unit, a first transmission unit, and a second reception unit.
The device outside the vehicle includes a first reception unit, a second determination unit, and a second transmission unit.

The acquisition unit acquires ambient information that is information indicating a situation around the vehicle (S110).
The detection unit detects an image target that is a target on the image by information processing from the surrounding information acquired by the acquisition unit, a target type that is the type of the target, and a target that is the size of the target. The size and the target position which is the position of the target are detected (S120).

  When the target type detected by the detection unit corresponds to a preset type, the first determination unit evaluates a type detection result that is a detection result of the target type by the detection unit, and determines the type detection result. If the evaluation value, which is the evaluated value, is greater than or equal to a preset threshold value, it is determined that the reliability of the type detection result is high. If the evaluation value is less than the set threshold value, the reliability of the type detection result is determined. It is determined that the property is low (S130, S140, S150).

  The first generation unit includes target information including a target type, a target size, and a target position as information on the image target when the first determination unit determines that the reliability of the type detection result is high. Is generated (S200).

The execution unit performs driving support by controlling various in-vehicle devices according to the target information generated by the first generation unit (S210).
If the first determination unit determines that the reliability of the type detection result is low, the second generation unit generates partial information that is a portion including the image target in the surrounding information (S160).

The first transmission unit transmits the partial information generated by the second generation unit to the external device (S170).
The first receiving unit receives the partial information transmitted from the in-vehicle device (S310).

The second determination unit determines whether the type detection result is appropriate based on the partial information received by the first reception unit (S320).
A 2nd transmission part transmits the determination result by a 2nd determination part to vehicle equipment as determination information (S330).

The second receiving unit receives the determination information transmitted from the external device (S180).
When the determination information received by the second reception unit indicates that the type detection result is appropriate when the first determination unit determines that the reliability of the type detection result is low, the first generation unit, The target information is generated (S190, S200).

  According to such a configuration, when the reliability of the type detection result is high, the in-vehicle device generates target information as information on the image target, and controls various in-vehicle devices according to the target information for operation. Provide support. When the reliability of the type detection result is low, the partial information is transmitted to the external device, the suitability of the type detection result is determined by the external device, and the determination information indicating the determination result is returned to the in-vehicle device. On the other hand, in the in-vehicle device, when the determination information received that the type detection result is appropriate indicates, target information is generated, and various in-vehicle devices are controlled according to the target information to provide driving support. .

  In this way, when the reliability of the type detection result is low, the suitability of the type detection result is determined by the external device, so that the reliability of the type detection result can be improved. Further, communication between the in-vehicle device and the external device is performed only when the reliability of the type detection result is low, and the communication load of the driving support system is suppressed.

  Therefore, when driving support is performed using the target detection result that is the result of detecting the target from the surrounding information that is information indicating the situation around the vehicle, the target detection result while suppressing the communication burden of the system Reliability can be improved.

  In addition, the code | symbol in the parenthesis described in this column and a claim shows the correspondence with the specific means as described in embodiment mentioned later as one aspect, Comprising: The technical scope of this invention is shown. It is not limited.

It is a block diagram which shows the structure of a driving assistance system. It is explanatory drawing which shows the attachment position of the image camera to the own vehicle, and the captured image imaged with the image camera. It is explanatory drawing of a driving assistance process. It is a flowchart of a driving assistance process. It is a flowchart of an apparatus side auxiliary process. It is explanatory drawing of a driving assistance process. It is explanatory drawing of a driving assistance process. It is explanatory drawing of a driving assistance process. It is explanatory drawing of a driving assistance process.

Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings.
[1. overall structure]
As shown in FIG. 1, the driving support system 1 of this embodiment includes an in-vehicle device 10 and an external device 20. The in-vehicle device 10 is a device mounted on the host vehicle 100. The off-vehicle device 20 is a device provided outside the host vehicle 100 such as a management center or a cloud server. The in-vehicle device 10 and the external device 20 can communicate with each other wirelessly.

[1.1. Configuration of in-vehicle device 10]
The in-vehicle device 10 includes a sensor 11, an information processing unit 12, an execution unit 13, and a communication unit 14. The information processing unit 12 is communicably connected to the sensor 11, the execution unit 13, and the communication unit 14.

  Specific examples of the sensor 11 include an image sensor, a stereo camera, a laser radar, and a millimeter wave radar. In the following description, an example in which an image sensor is employed as the sensor 11 will be described.

  The sensor 11 is disposed near the center of the host vehicle 100. The sensor 11 images the surroundings of the host vehicle 100. FIG. 2 shows a position where the sensor 11 is attached to the host vehicle 100. In the present embodiment, the sensor 11 images a search area set in the host vehicle 100. As shown in FIG. 2, the sensor 11 detects a target from the captured image by performing known image processing such as template matching on the captured data representing the captured image that is the captured image. Examples of target types include vehicles, white lines, and passers-by.

  The information processing unit 12 is mainly configured by a known microcomputer having a CPU 12A and a semiconductor memory such as a RAM 12B, a ROM 12C, and a flash memory 12D. Various functions of the information processing unit 12 are realized by the CPU 12A executing a program stored in a non-transitional tangible recording medium. In this example, the semiconductor memory corresponds to a non-transitional tangible recording medium storing a program. Further, by executing this program, a method corresponding to the program is executed. Note that the number of microcomputers constituting the information processing unit 12 may be one or more.

  The execution unit 13 controls various in-vehicle devices and executes predetermined driving assistance. The execution unit 13 includes at least a monitor that displays various images and a speaker that outputs an alarm sound and guidance sound as in-vehicle devices to be controlled. The execution unit 13 may further include various control devices that control an internal combustion engine, a power train mechanism, a brake mechanism, and the like mounted on the host vehicle 100 as in-vehicle devices to be controlled.

The communication unit 14 transmits and receives various data wirelessly with the communication unit 21 of the vehicle exterior device 20.
[1.2. Configuration of External Device 20]
The vehicle exterior device 20 includes a communication unit 21 and an information processing unit 22. The communication unit 21 and the information processing unit 22 are connected to be communicable with each other.

The communication unit 21 transmits and receives various data wirelessly with the communication unit 14 of the in-vehicle device 10.
The information processing unit 22 is configured around a known microcomputer having a CPU 22A and a semiconductor memory such as a RAM 22B, a ROM 22C, and a flash memory 22D. Various functions of the information processing unit 22 are realized by the CPU 22A executing a program stored in a non-transitional tangible recording medium. In this example, the semiconductor memory corresponds to a non-transitional tangible recording medium storing a program. Further, by executing this program, a method corresponding to the program is executed. Note that the number of microcomputers constituting the information processing unit 22 may be one or plural.

[2. processing]
Various processes executed by the driving support system 1 will be described.
[2.1. Driving support processing]
The driving support process executed by the in-vehicle device 10 will be described with reference to the flowchart of FIG.

In the first step S <b> 110, the sensor 11 images the search area set in the host vehicle 100. Thereafter, the process proceeds to S120.
In S120, the sensor 11 performs well-known image processing such as template matching on the imaging data representing the captured image, which is the captured image, thereby obtaining the image target that is the target on the image from the captured image. It is detected together with the target type that is the type, the target size that is the size, and the target position that is the position. The captured image is an example of ambient information that is information indicating a situation around the vehicle. FIG. 2 shows an example in which a target is detected from a captured image. In this example, passersby are detected at six locations. Of these, four people correctly detect passersby. In addition, there are two places where no passerby is detected as a passerby. There is one place where no passerby is detected. In the example illustrated in FIG. 6, the target “1” and the target “2” are detected from the captured image. Then, the sensor 11 transmits image target information, which is information regarding the image target detected by this process, to the information processing unit 12. The image target information includes information about the target type, target size, and target position. Thereafter, the process proceeds to S130.

  In S <b> 130, the information processing unit 12 determines whether the target type included in the image target information corresponds to a preset type. In the present embodiment, a passerby is set for the type. If it is determined that the target type corresponds to a preset type, the process proceeds to S140. If it is determined that the target type does not correspond to the preset type, this process is terminated.

In S140, the information processing unit 12 determines the reliability of the type detection result that is the detection result of the target type. FIG. 3 shows an example of determining the reliability of the type detection result. The information processing unit 12 compares the image target detected as the passer-by with the passer-by model, thereby calculating a score value indicating the passer-byness of the image target detected as the passer-by. The score value corresponds to an evaluation value that is a value obtained by evaluating the type detection result. In the passer-by model, a number of simplified shapes of passers-by are registered. When the image target and the shape registered in the passer-by model are approximated, a value corresponding to the degree of approximation is added to the score value. When the image target and the shape registered in the passerby model are not approximate, a value corresponding to the degree is subtracted from the score value. If the score value is equal to or greater than a preset determination threshold, the information processing unit 12 determines that the reliability of the type detection result is high. In the present embodiment, the determination threshold is set to a numerical value “0”. When the score value is less than the determination threshold, the information processing unit 12 determines that the reliability of the type detection result is low. In the example illustrated in FIG. 6, it is determined that the reliability of the type detection results of the target “1” and the target “2” is low. Thereafter, the process proceeds to S150.

  In S150, the information processing unit 12 determines whether the reliability of the type detection result is high. When it is determined that the reliability of the type detection result is high, the process proceeds to S200. When it is determined that the reliability of the type detection result is low, the process proceeds to S160.

  In S160, the information processing unit 12 generates a target image that is a portion of the captured image that includes the image target. A target image is an example of partial information that is a portion of surrounding information that includes an image target. In the present embodiment, when the information processing unit 12 generates a target image, the information processing unit 12 deletes a portion of the captured image that overlaps the target image that has been transmitted to the vehicle exterior device 20.

  FIG. 7 shows an example in which BOX1 and BOX2 partially overlap as case 1, an example in which BOX2 is included in BOX1 as case 2, and an example in which BOX1, BOX2 and BOX3 partially overlap as case 3. Yes. BOX1, BOX2, and BOX3 correspond to target images. Here, it is assumed that images are captured in the order of BOX1, BOX2, and BOX3.

  In case 1, as shown in FIG. 8, for BOX1, the entire BOX1 is generated as a target image. The image data indicating the BOX 1 includes basic information that is information regarding the BOX. The basic information includes an image number, an image height, an image width, a code indicating the head of data, and a data body. As for BOX2, a portion of BOX2 from which a portion overlapping with BOX1 is deleted is generated as a target image. The image data indicating the BOX 2 includes basic information regarding the BOX 2 and duplication related information. Duplicate related information is information about a portion where BOX overlaps, a code indicating the head of the duplicate related information, the number of the duplicate destination, the number of corners of the duplicate portion, the address of each corner of the duplicate portion at the duplicate destination , The address of each corner of the overlapping portion in the image is included.

In Case 2, as shown in FIG. 8, BOX1 and BOX2 are the same as Case 1.
In Case 3, as shown in FIG. 9, BOX1 and BOX2 are the same as Case 1. As for BOX3, a portion of BOX3 from which a portion overlapping with BOX1 or BOX2 is deleted is generated as a target image. The image data indicating BOX 3 includes basic information related to BOX 3 and duplication related information related to duplication with other BOX. That is, when overlapping with N other BOXes, overlap related information corresponding to each of the N other BOXes is included. As described above, the target image is generated so that the overlapping portion is transmitted only once.

In the example illustrated in FIG. 6, target images are generated for the target “1” and the target “2” that are determined to have low reliability in the recognition result of the target type.
Thereafter, the process proceeds to S170.

  In S <b> 170, the communication unit 14 transmits the target image generated by the information processing unit 12 to the vehicle exterior device 20. In the example illustrated in FIG. 6, target images for the target “1” and the target “2” are transmitted to the vehicle exterior device 20. Thereafter, the process proceeds to S180.

In S180, the communication unit 14 determines whether or not the determination information transmitted from the vehicle exterior device 20 has been received. The determination information is information indicating whether or not the type detection result is appropriate. When it is determined that the determination information has not been received, the process waits until the determination information is received by executing S180 again. If it is determined that the determination information has been received, the process proceeds to S190.

  In S190, the information processing unit 12 determines whether or not the type detection result is appropriate based on the determination result indicated in the determination information received by the communication unit 14. In this embodiment, the information processing unit 12 determines that the type detection result is appropriate when the determination information is the symbol “1”, and detects the type when the determination information is the symbol “0”. Judge that the result is not appropriate. When it is determined that the type detection result is appropriate, the process proceeds to S200. If it is determined that the type detection result is not appropriate, this processing is terminated as it is.

In S200, the information processing unit 12 generates target information including a target type, a target size, and a target position as information on the image target. Thereafter, the process proceeds to S210.
In S <b> 210, the execution unit 13 controls various in-vehicle devices according to the target information created by the information processing unit 12 and executes predetermined driving assistance. Thereafter, this process is terminated.

[2.2. Device side auxiliary processing]
Device side auxiliary processing executed by the vehicle exterior device 20 will be described with reference to the flowchart of FIG.

  First, in step S310, the communication unit 21 determines whether or not the target image transmitted from the in-vehicle device 10 has been received. If it is determined that the target image has not been received, the process waits until the target image is received by executing S310 again. If it is determined that the target image has been received, the process proceeds to S320.

  In S320, the information processing unit 22 determines whether the type detection result is appropriate based on the target image received by the communication unit 21. Here, the method described in S130 is used. However, compared to the information processing unit 12, the information processing unit 22 has abundant resources allocated to processing, has a high processing speed, has a fine shape registered in a passer-by model, and has a large quantity. Have. If the score value is greater than or equal to a preset determination threshold, the information processing unit 22 determines that the type detection result is appropriate. When the score value is less than the determination threshold, the information processing unit 22 determines that the type detection result is not appropriate. In the example illustrated in FIG. 6, when it is determined that the type detection result is appropriate, the symbol “1” is added as determination information indicating the determination result, and the type detection result is determined to be inappropriate. Is assigned a symbol “0” as determination information indicating the determination result. Symbol “1” and symbol “0” are examples of symbols preset as determination information. Thereafter, the process proceeds to S330.

In S <b> 330, the communication unit 21 transmits determination information expressed by a symbol set in advance as a determination result by the information processing unit 22 to the in-vehicle device 10. Thereafter, this process is terminated.
[2.3. Correspondence with Claims]
S110 corresponds to the acquisition unit. S120 corresponds to the detection unit. S130, S140, and S150 correspond to the first determination unit. S160 corresponds to the first generation unit. S170 corresponds to the first transmission unit. S180 corresponds to the second receiving unit. S190 and S200 correspond to the second generation unit. S210 corresponds to the execution unit. S310 corresponds to the second receiving unit. S320 corresponds to the second determination unit. S330 corresponds to the second transmission unit.

[3. effect]
According to the first embodiment described in detail above, the following effects can be obtained.
(1a) When the reliability of the type detection result is high, the in-vehicle device 10 generates target information as information on the image target, and performs driving support by controlling various in-vehicle devices according to the target information. If the reliability of the type detection result is low, the target image is transmitted to the outside device 20, the suitability of the type detection result is determined by the outside device 20, and the determination information indicating the determination result is returned to the in-vehicle device 10. Get it. The in-vehicle device 10 generates target information when the received determination information indicates that the type detection result is appropriate, and performs driving support by controlling various in-vehicle devices according to the target information.

  Thus, when the reliability of the type detection result is low, the suitability of the type detection result is determined by the out-of-vehicle device 20, so that the reliability of the type detection result can be improved. Further, the communication between the in-vehicle device 10 and the external device 20 is performed only when the reliability of the type detection result is low, and the communication load of the driving support system 1 is suppressed.

  Therefore, when performing driving support using the target detection result that is the result of detecting the target from the ambient information that is information indicating the situation around the vehicle, the communication burden of the driving support system 1 is suppressed while suppressing the communication load. The reliability of the target detection result can be improved.

  (1b) When the information processing unit 12 generates the target image, the information processing unit 12 deletes a portion of the captured image that overlaps the target image that has already been transmitted to the vehicle exterior device 20. For this reason, the communication burden of the driving assistance system 1 can be further suppressed.

  (1c) The communication unit 21 transmits determination information expressed by a symbol set in advance as a determination result by the information processing unit 22 to the in-vehicle device 10. For this reason, the communication burden of the driving assistance system 1 can be further suppressed.

[4. Other Embodiments]
As mentioned above, although the form for implementing this invention was demonstrated, this invention is not limited to the above-mentioned embodiment, It can implement in various deformation | transformation.

  (3a) A plurality of functions of one constituent element in the above embodiment may be realized by a plurality of constituent elements, or a single function of one constituent element may be realized by a plurality of constituent elements. . Further, a plurality of functions possessed by a plurality of constituent elements may be realized by one constituent element, or one function realized by a plurality of constituent elements may be realized by one constituent element. Moreover, you may abbreviate | omit a part of structure of the said embodiment. In addition, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified only by the wording described in the claim are embodiment of this invention.

  (3b) In addition to the above-described driving support system 1, a program for causing a computer to function as each of the in-vehicle device 10 and the external device 20, and the in-vehicle device 10 and the external device 20 constituting the driving support system 1, and recording this program The present invention can also be realized in various forms such as a non-transitional actual recording medium such as a semiconductor memory, a driving support method, and a device side auxiliary method.

  DESCRIPTION OF SYMBOLS 1 ... Driving assistance system, 10 ... In-vehicle apparatus, 11 ... Sensor, 12 ... Information processing part, 12A ... CPU, 12B ... RAM, 12C ... ROM, 12D ... Flash memory, 13 ... Execution part, 14 ... Communication part, 20 ... External equipment, 21 ... communication unit, 22 ... information processing unit, 22A ... CPU, 22B ... RAM, 22C ... ROM, 22D ... flash memory, 100 ... own vehicle.

Claims (4)

A driving support system (1),
An in-vehicle device (10) and an external device (20),
The in-vehicle device is a device mounted on the vehicle (100), the out-of-vehicle device is a device provided outside the vehicle, and the in-vehicle device and the out-of-vehicle device can communicate wirelessly,
The in-vehicle device is
An acquisition unit (S110) that acquires a captured image, which is an image obtained by capturing the situation around the vehicle;
An image target that is a target on an image by image processing from the captured image acquired by the acquisition unit, a target type that is a type of the target, a target size that is a size of the target, A detection unit (S120) that detects the target position that is the position of the target;
When the target type detected by the detection unit corresponds to a preset type, the type detection result that is the detection result of the target type by the detection unit is evaluated, and the type detection result is evaluated When the evaluation value, which is a value, is greater than or equal to a preset threshold value, it is determined that the type detection result is highly reliable, and when the evaluation value is less than the set threshold value, the type detection result A first determination unit (S130, S140, S150) that determines that the reliability of the
Target information including the target type, the target size, and the target position as information about the image target when the first determination unit determines that the reliability of the type detection result is high A first generation unit (S200) for generating
In accordance with the target information generated by the first generation unit, an execution unit (S210) that performs driving support by controlling various in-vehicle devices,
A second generation unit (S160) that generates a target image that is a portion including the image target in the captured image when the first determination unit determines that the reliability of the type detection result is low. When,
A first transmission unit (S170) that transmits the target image generated by the second generation unit to the vehicle exterior device,
The outside equipment is
A first receiving unit (S310) for receiving the target image transmitted from the in-vehicle device;
A second determination unit (S320) that determines the suitability of the type detection result based on the target image received by the first reception unit;
A second transmission unit (S330) that transmits the determination result by the second determination unit to the in-vehicle device as determination information,
The in-vehicle device further includes
A second receiving unit (S180) for receiving the determination information transmitted from the vehicle equipment;
The first generation unit, when the first determination unit determines that the reliability of the type detection result is low, the determination received by the second reception unit that the type detection result is appropriate When the information indicates, the target information is generated (S190, S200).
Driving support system. The driving support system according to claim 1,
The second generating unit is the target when generating the image, image data representing the target object image of the transmitted of the target portion to delete the portion overlapping with the image on the outside apparatus out of the captured image The image data includes duplication related information that is information relating to an overlapping portion, and the duplication related information includes a code indicating a head of the duplication related information, a duplication destination number, and a corner of the duplication portion. The driving support system includes the number of parts, the address of each corner of the overlapping part at the overlapping destination, and the address of each corner of the overlapping part in the target image .   The driving support system according to claim 1 or 2,
  The in-vehicle device communicates with the external device only when the reliability of the type detection result is low.
  Driving support system. The driving support system according to any one of claims 1 to 3 , wherein
The second transmission unit transmits the determination information expressing the determination result with a preset symbol to the in-vehicle device.

Images