WO2009070069A1

WO2009070069A1 - A system for classifying objects in the vicinity of a vehicle

Info

Publication number: WO2009070069A1
Application number: PCT/SE2007/050901
Authority: WO
Inventors: Ognjan Hedberg; Jonas HAMMARSTRÖM
Original assignee: Autoliv Development Ab
Priority date: 2007-11-26
Filing date: 2007-11-26
Publication date: 2009-06-04
Also published as: EP2212160A1; EP2212160A4

Abstract

A classification system for classifying objects in the vicinity of a vehicle, the system comprising: a video camera (1) for gathering camera data; a reflected radiation system (2) for gathering reflected radiation data; and a classifier (9), wherein raw data from the camera (1) and the reflected radiation system (2) are combined and analysed by the classifier (9), the classifier (9) being configured to provide an output relating to the type of an object (6) that appears in data gathered by both the camera (1) and the reflected radiation system (2).

Description

A system for classifying objects in the vicinity of a vehicle.

Description of invention

THIS INVENTION relates to a classification system, and in particular concerns a system for classifying objects in the vicinity of a vehicle.

Modern motor vehicles are typically equipped with several different safety systems, which are adapted to protect both occupants of the vehicle (in the case of internal air- bags or seal belt pretεnsioners), or pedestrians (for example, bonnet lifters and external air-bag.

When it is determined by vehicle crash sensors that an impact is imminent, or that crash is occurring, one or more of these safety systems may be deployed, or a safety system may be deployed in one of a plurality of possible modes, depending in part upon the type of the other object that is involved. For instance, an impact with a pole or tree at a given speed may be more severe than in impact with another vehicle. If it appears that the vehicle is about to strike a pedestrian, then an external air-bag or bonnet lifter may be activated, but if the vehicle is about to strike an inanimate object such as a tree then there is no need for these protection systems to be deployed.

Accurate classification of objects in the vicinity of a vehicle is therefore important for vehicle safety systems to be activated in the most appropriate manner.

Existing object classifiers comprise computer programs which are operable to analyse data from a vehicle sensor, such as a camera or radar system. The classifier is "trained^'5 with exposure to many different types of object in different circumstances, so that the program is able to make an accurate determination as to the type of a new kind of object that is detected,

It is an object of the present invention to provide an improve classification system of this type. Accordingly, one aspect of the present invention provides a classification system for classifying objects in the vicinity of a vehicle, the system comprising: a video camera for gathering camera data; a reflected radiation system for gathering reflected radiation data; and a classifier, wherein raw data from the camera and the reflected radiation system are combined and analysed by the classifier, the classifier being configured to provide an output relating to the type of an object that appears in data gathered by both the camera and the reflected radiation system,

Advantageously, the determination comprises an indication that the object is a certain type of object.

Alternatively, the determination comprises a probability that the object is of a certain type,

Preferably, the classifier comprises a neural network, a support vector machine or an adaptive boosting classifier.

Conveniently, the classifier is trained to discriminate between different types of object.

Advantageously, the classifier is trained to discriminate between pedestrians and other types of object.

Preferably, the classifier is trained to discriminate between pedestrians, vehicles and pole- like objects.

Conveniently, the output of the classifier is used to control the deployment of a pedestrian protection device,

Advantageously, the output of the classifier is used to control the deployment of both a pedestrian protection device and an occupant protection device. Preferably, the data gathered by the camera and the reflected radiation system are input into a pre-selector., which is configured to analyse the data from the camera and the reflected radiation system and to identify regions of the data which contain an object of potential interest.

Conveniently, only the regions identified by the pre-selector are analysed by the

Advantageously, the data from the camera and/or the reflected radiation system is analysed by an impact evaluator, which evaluates the risk of an impact between the vehicle and the object of potential interest.

Preferably, outputs from both the classifier and the impact evaluator are used to control the deployment of one or more protection devices.

Conveniently, the protection devices are pedestrian protection devices and/or occupant protection devices.

Advantageously, the reflected radiation system is a radar system.

In order that the present invention may be more readily understood, embodiments thereof will now be described, by way of example, with reference to the accompanying drawings, in which:

Figures 1 and 4 show schematic views of the components of a classification system embodying the present invention; and

Figures 2 and 3 show schematic views of data gathered by different on-board vehicle sensors. Referring firstly to figure I₅ components of a system embodying the present invention are shown. A vehicle camera 1 and a radar system 2 are both provided, being mounted on a vehicle so that the fields of view of the camera 1 and radar system 2 are overlapping,

The camera 1 may be a "mono" camera, may comprise a stereo camera system, may comprise an infrared (IR) camera or be any other suitable kind of camera that forms an image from received light, which need not fall within the visible portion of the spectrum.

The radar system 2 may be a conventional radar system, or may alternatively comprise a lidar system, an ultrasonic system or any other suitable system in which waves are emitted by an emitter and reflected waves which return to the vehicle are analysed as will be understood by those of skill in the art.

Data from both the camera 1 and the radar system 2 are output to a pre-selector 3, which identifies potential objects of interest in the data.

Referring to figure 2, simplified images 4,5 from the camera system 1 and the radar system 2 are shown, in the case where a pedestrian 6 is positioned in front of the vehicle. The image 4 from the camera 1 comprises an image of the pedestrian 6, whereas the image 5 from the radar system 2 indicates that, in the same region of space, there is an object which is closer to the vehicle than other detected '"background" objects,

The pre-classifϊεr 3 is operable to determine that the data from the camera 1 and radar system 2 contain an object of potential interest, and corresponding regions 7,8 of the images 4,5 taken by the camera i and radar system 2 are identified by the pre~selector for further analysis. These regions 7,8 of the images 4,5 are the regions containing the data relating to the object of potential interest. Returning to figure 1, data from the selected regions 7,8 of the images 4,5 is combined, and the combined camera and radar data is then passed to a trained classifier 9, As discussed above, an object classifier is trained through exposure to different types of objects in different circumstances, so that the classifier is able to provide a high degree of accuracy in classifying objects in new data that is presented. Sn this case, the classifier 9 is trained by repeated exposure to combined camera and radar data relating to different types of object.

The classifier 9 analyses the combined camera and radar data io provide a determination as to the type of object that appears in the combined data.

The skilled person will appreciate that different types of classifier might be used. Examples are neural network classifiers, support vector machine classifiers or Adaptive Boosting ("Adaboost") classifiers.

Examples of features in the combined raw camera/radar data that could be analysed to classify objects in the data include grey scale values, gradients, patterns, amplitudes and (for the radar data only) phase information.

The classifier 9 may analyse the data to determine whether it is one of several types of essential object in a single step. Alternatively, as shown in figure 4, the classifier 9 may analyse the raw combined camera/radar data separately for each potential type of object, in parallel. Referring to figure 4, the raw data is input to a pedestrian classifier 10, a pole classifier 11 and a vehicle classifier 12, and each classifier analyses the data and provides a determination as to whether the data contains an object of that particular classification, or alternatively outputs a measure of probability that the data contains that type of object.

In preferred embodiments, each classifier 10,11,12 is operable to output a "negative" signal if it is determined that the data does not contain an object of that particular type.

If all of the classifiers 10.11,12 out negative signals, this may be passed to an error generator 13, which outputs a signal indicating that the classifier 9 has not been able to identify the object. In this case, the system may default to a "safest^"' mode in which both occupant and pedestrian safety systems are activated if activation would be triggered by the object being of the most hazardous type.

The combined camera/radar data is also input to an impact evaluator 14, which analyses the data in parallel with the classifier 9. The impact evaluator 14 analyses the data to calculate the likelihood of the vehicle being involved in an impact with the object in question, and will also make a determination as to the likely time of the impact, and the relative speed and/or orientation of the vehicle and the object at the predicted impact. Instead of analysing the combined data, in alternative embodiments the impact evaluator 14 may consider only the camera data, or only the radar data. Indeed, the impact evaluator 14 may take data from a further sensor 17, such as an accelerometer or impact sensor, alone or in combination with the camera and/or radar data.

In preferred embodiments, only data from the camera 1 and radar system 2 in one or more regions identified by the pre-classifier 3 as containing an object of potential interest are analysed by the classifier 9. The pre-classifier 3 may discard other data and only pass on data relating to these regions, or the data relating to these regions may be "marked" as being for analysis.

Outputs from the classifier 9 and the impact evaluator 14 are then passed to an algorithm 15 for pedestrian protection, and also to an algorithm 16 for occupant protection. These algorithms 15,16 will coordinate the deployment, if appropriate, of one or more safety system to protect pedestrians and/or vehicle occupants, in dependence upon the outputs from the classifier 9, which provides an indication of the type of object, and from the impact evaluator 14.

The use of combined camera and radar data for analysis in a trained classifier will allow more accurate and robust classification of objects than analysis of either of camera or radar data in isolation. Depending on the particular circumstances, an object may have more distinctive characteristics when camera data or radar data containing the object are considered, and combining these two types of data will ensure that the most clearly defining characteristic of a certain object can always be taken into account.

When used in this specification and claims, the terms "comprises" and "comprising" and variations thereof mean that the specified features, steps or integers are included. The terms are not to be interpreted to exclude the presence of other features, steps or components.

The features disclosed in the foregoing description, or the following claims, or the accompanying drawings, expressed in their specific forms or in terms of a means for performing the disclosed function, or a method or process for attaining the disclosed result, as appropriate, may, separately, or in any combination of such features, be utilised for realising the invention in diverse forms thereof.

Claims

CLAIMS;

1. A classification system for classifying objects in the vicinity of a vehicle, the system comprising: a video camera { 1) for gathering camera data; a reflected radiation system (2) for gathering reflected radiation data; and a classifier (9), wherein raw data from the camera (1) and the reflective radiation system (2) are combined and analysed by the classifier (9), the classifier (9) being configured to provide an output relating to the type of an object (6) that appears in data gathered by both the camera (1) and the reflected radiation system (2).

2, A system according to claim 1, wherein the determination comprises an indication that the object (6) is a certain type of object,

3. A system according to claim 1, wherein the determination comprises a probability that the object (6) is of a certain type.

4. A system according to any preceding claim, wherein the classifier (9) comprises a neural network, a support vector machine or an adaptive boosting classifier.

5. A system according to any preceding claim, wherein the classifier (9) is trained to discriminate between different types of object.

6. A classifier according to claim 5, wherein the classifier (9) is trained to discriminate between pedestrians and other types of object.

7. A classifier according to claim 5 or 6, wherein the classifier (9) is trained to discriminate between pedestrians, vehicles and pole-like objects.

8. A system according to any preceding claim, wherein the output of the classifier (9) is used to control the deployment of a pedestrian protection device.

9. Λ system according to claim 8, wherein the output of the classifier (9) is used to control the deployment of both a pedestrian protection device and an occupant protection device.

10. Λ system according to any preceding claim, wherein the data gathered by the camera (I) and the reflected radiation system (2) are input into a pre-selector, which is configured to analyse the data from the camera and the reflected radiation system (2) and to identify regions of the data which contain an object of potential interest,

11. A system according to claim 10, wherein only ihe regions identified b> the preselector are analysed by the classifier (9).

12. A system according to any preceding claim, wherein the data from the camera (1) and/or the reflected radiation system (2) is analysed by an impact evaluator (14), which evaluates the risk of an impact between the vehicle and the object of potential interest.

13. A system according to claim 12, wherein outputs from both the classifier (9) and the impact evaluator (14) arc used to control the deployment of one or more protection devices.

14. A system according to claim 13, wherein the protection devices are pedestrian protection devices and/or occupant protection devices.

15. A system according to any preceding claim, wherein the reflected radiation system is a radar system.