ES2363962B1 - PICTURE ACQUISITION SYSTEM FOR APPLICATIONS OF DRIVING ASSISTANCE OF AUTOMATIC AUTOMATIC VEHICLES AUTOMATICALLY BASED ON THE DRIVING ENVIRONMENT AND THE VEHICLE OWN. - Google Patents

Abstract

Image acquisition system for driving assistance applications of motor vehicles automatically adaptable according to the driving environment and the vehicle itself, comprising: # - a variable-focus optics camera (1) installed in the vehicle in charge of capturing the images; # - a control module (2) of the camera (1) configured to receive from each driving assistance application · (3), a status information indicating whether the application (3) is active or not and an optimum operating range of the camera's viewing angle (1) and, depending on that data, adjust the camera's viewing angle (1) to optimize the operation of the vision applications (3) that are active . # The driving assistance applications (3) and the control module (2) can receive information from a plurality of sensors (4) installed in the vehicle.

Description

Image acquisition system for driving assistance applications of automatically adaptable motor vehicles based on the driving environment and the vehicle itself.

Field of the Invention

The present invention has as its main objective its application in the development and use in vision-based driving assistance systems.

Background of the invention

In the development of vision-based driving assistance systems, one of the most important and determining elements to evaluate is the camera or cameras that will be used to acquire the images. Properly selecting the solution that best meets the system requirements is essential for good results. In general, we can distinguish, for any camera used in driving assistance applications, three characteristics that describe them: optics, sensor and connection.

Optics is the set of lenses that concentrate the rays of visible light and project them on the image sensor. One of the most important parameters of an optics is the focal length. This value determines the viewing angle the camera sees.

The sensor and the electronics that accompany it, are the elements of the chamber responsible for transforming the radiation of visible light into an electrical signal. This signal is adapted so that it can later be transmitted to the PC

or hardware platform. There are many features that define a sensor, which should be taken into account when choosing the most suitable for each application.

As for the connection, the camera must be able to be connected to a PC or hardware platform where the images to be processed are received. Currently there are different connection solutions: USB, Ethernet, LVDS, etc. Your choice will depend on the needs of each application. For some camera models, the connection must also allow communication in the opposite direction, in order to send control information to the camera.

The focal length, or focal length of a lens, is a very important parameter to consider when selecting the optics to use. This value is described as the distance between the optical center of the lens and its focus (or focal point). The focal length and image sensor size determine the observed viewing angle (FOV), which can be obtained with the following equation:

Horizontal FOV = atan ((sensorSizeH / 2) / focalLength)

Vertical FOV = atan ((sensorSizeV / 2) / focalLength)

Where focalLenght is the focal length and sensorSizeH and sensorSizeV are the horizontal and vertical dimensions of the sensor respectively. The dimensions of the sensor are obtained from two values, which are pixel size and resolution.

The pixel size is precisely the actual size that the area dedicated to storing the image information equivalent to a pixel occupies in the sensor. The resolution indicates the number of horizontal and vertical pixels of the image delivered by the sensor. The following table, Table 1, shows an example of a sensor with a pixel size of 6 microns and a resolution of 752x480 pixels.

TABLE 1

Calculation of actual physical dimensions of a sensor

The sensors are usually rectangular, so the horizontal and vertical viewing angles are different. Figure 1 shows the area of incidence of the image projected by the optics (striped circle) on the sensor (rectangle). We see that to cover the entire sensor the image area from the optics is larger than that of the sensor. In the following table, Table 2, we can see some viewing angles for different focal points.

TABLE 2

Examples of viewing angles based on the focal length calculated for the sensor in the previous example (6 microns and 752x480)

The viewing angle (FOV) determines how we see the information present in the scene. With a larger viewing angle we will have more extensive information of the scene, but as a counterpart, the objects will be smaller. If we have a smaller viewing angle, we will lose the information of the objects located on our sides, but we will see objects located in the distance closer.

When developing vision-based applications, choosing which optics to use will always be a compromise decision. This is because the intention is to be able to make the greatest number of driving assistance applications using images from the same sensor. But depending on the application we know that in some of them it would be desirable to have a greater focal length and in others less. For example, applications such as pedestrian detection will prefer a higher FOV optics in order to have a better perimeter view around our vehicle, while for applications such as vehicle detection they will work better with lower FOV optics, which allow distant objects to be seen with larger size

In the same way, for the same application it would also be desirable to be able to modify the FOV according to the conditions in which we are. For example, applications could make variations of the FOV depending on the speed at which we travel with the vehicle or depending on the angle of rotation of the steering wheel. Figure 2 shows examples of vision applications to detect pedestrians and to detect vehicles. For vehicle detection, optics with a lower viewing angle would allow you to see the nearest vehicles and improve detection performance. In the case of vehicle detection, it is important that at high speeds we reduce the FOV to be able to focus on the farthest vehicles, and to expand this value as the speed is reduced as the information on the sides begins to be more relevant, to Pedestrian detection.

Most vision applications can be deactivated depending on several factors, such as being in non-functioning speed ranges, disabling per user or due to problems in image processing algorithms. This state of disconnection is known by each application, so, in any of these circumstances, the position of the optics could be adapted in the most suitable state for those applications that are active.

The present invention uses cameras with real-time variable focus optics in automotive applications, in the field of driving assistance, so that it is possible to simultaneously implement different applications with different requirements. A system such as vehicle recognition, for example, requires a high detection and tracking distance, which is greater as the speed increases. In this case, the focal length of the camera could increase in order to obtain more information from distant objects. On the contrary, at low speed, long distances are not necessary and the lateral field begins to be important. This also happens in other applications such as the detection of pedestrians, with greater importance of lateral field, or the detection of lane lines, where the distance takes on an important value.

Description of the invention

The invention relates to an image acquisition system for driving assistance applications of motor vehicles, automatically adaptable according to the driving environment and the vehicle itself, according to claim 1. Preferred embodiments of the system are defined in the dependent claims.

The system includes:

-

a variable focal optics camera installed in the vehicle responsible for capturing the images;

-

a camera control module configured to receive from each driving assistance application, installed in the vehicle and operating by processing the images delivered by said camera, status information indicating whether the application is active or not and a range Optimum operation of the camera's viewing angle and, depending on that data, adjust the camera's viewing angle to optimize the operation of active vision applications.

The control module is preferably also configured to receive information from a plurality of sensors installed in the vehicle and use said information, together with that supplied by the driving assistance applications, to determine the camera's angle of view adjustment.

Driving assistance applications may be configured to receive information from a plurality of sensors installed in the vehicle and use said information to define at least one of the following:

-

the optimum operating range of camera viewing angle;

-

the status of the driving assistance application, active or not active.

Brief description of the drawings

A series of drawings that help to better understand the invention and that expressly relate to an embodiment of said invention which is presented as a non-limiting example thereof is described very briefly below.

Figure 1 represents the incidence of the image projected by the optics on the sensor.

Figure 2 shows different examples of vision applications.

Figure 3 represents an operating scheme of a camera system with variable focal optics.

Description of a preferred embodiment of the invention

The system object of the present invention presents a set of innovations grouped into two sections:

•

Use of a real-time variable focal lens optics

•

Use of a camera control / management module.

Firstly, cameras with real-time variable-focus optics are used. The variation of the focal length is done quickly and electronically controlled. The intention of using this type of optics is to be able to adapt the area of vision of the camera according to the conditions in which we are. In this way, different applications can be obtained and the effect of the existing commitment by the integration, under the same sensor, of different vision applications can be minimized.

Compared to systems with fixed operating characteristics, dynamically adaptable systems based on certain variables give better results. These systems try to adapt in the best possible way to the ideal detection conditions, to be able to deliver the variables measured in the ideal conditions. On the other hand, it is immediate to see that this class of systems has a greater complexity of implementation than the first, since it implies the additional implementation of decision-making algorithms for the adaptation part.

In the case that concerns us the complexity is even greater, since, on the one hand, each vision application must decide which is the best position of the optics according to its state, and on the other hand, having to merge the operation For several applications, it is necessary to check the state of the desired optics of each application and make a joint decision as optimal as possible. This logical part is implemented in what has previously been designated as control module 2 of the variable focal camera 1. This control module 2 receives status information from each vision application 3 and optionally information from other sensors 4 (to obtain some environment variables: CAN data, light collectors, GPS, etc). The joint analysis of this information will determine the decision of the state in which the optics should be, depending on the driving environment and the vehicle itself.

Figure 3 shows an operation scheme of a camera system with variable focal optics 1. Each vision application 3 sends its status information to the control module 2 which will decide on the optimal state in which the optics should be.

Claims (4)

1. Image acquisition system for driving assistance applications of automatically adaptable motor vehicles based on the driving environment and the vehicle itself, characterized in that it comprises:

-

a variable focal optics camera (1) installed in the vehicle responsible for capturing the images;

-

a camera control module (2) (1) configured to receive from each driving assistance application

(3) a status information indicating whether the application (3) is active or not and an optimal operating range of the camera's viewing angle (1) and, depending on that data, adjust the viewing angle of the camera (1) to optimize the operation of vision applications (3) that are active.

2.

Image acquisition system according to claim 1, characterized in that the control module (2) is additionally configured to receive information from a plurality of sensors (4) installed in the vehicle and use said information, together with that supplied by the assistance applications to the conduction (3), to determine the adjustment of the viewing angle of the camera (1).

3.

Image acquisition system according to any of the preceding claims, characterized in that the driving assistance applications (3) are additionally configured to receive information from a plurality of sensors (4) installed in the vehicle and use said information to define at least one of the following:

-

the optimum operating range of camera viewing angle (1);

-

the status of the driving assistance application (3).

SPANISH OFFICE OF THE PATENTS AND BRAND Application no .: 200900019 SPAIN Date of submission of the application: 18.12.2008 Priority Date: REPORT ON THE STATE OF THE TECHNIQUE 51 Int. Cl.: G06T5 / 00 (2006.01) RELEVANT DOCUMENTS

Category

Documents cited Claims Affected

X

US 4926346 A (YOKOYAMA SHOUJI) 15.05.1990, column 2, lines 38-68; column 3; column 4, lines 1-24; column 8, lines 10-29. 1-3

X

ES 2286973 T3 (BAYERISCHE MOTOREN WERKE AG) 16.12.2007, column 1, lines 47-68; columns 2,3; column 4, lines 1-17; claims; figures. 1-3

X

DE 102004051527 A1 (DAIMLER CHRYSLER AG) 01.06.2006, the whole document. 1-3

X

US 5559695 A (DAILY MICHAEL J) 24.09.1996, the whole document. 1-3

Category of the documents cited X: of particular relevance Y: of particular relevance combined with other / s of the same category A: reflects the state of the art O: refers to unwritten disclosure P: published between the priority date and the date of priority submission of the application E: previous document, but published after the date of submission of the application

This report has been prepared • for all claims • for claims no:

Date of realization of the report 14.06.2011

Examiner M. González Vasserot Page 1/5

REPORT OF THE STATE OF THE TECHNIQUE Minimum documentation searched (classification system followed by classification symbols) G06T Electronic databases consulted during the search (name of the database and, if possible, terms of search used) INVENES, EPODOC, WPI  WRITTEN OPINION Date of Completion of Written Opinion: 06.16.2011 Statement

Novelty (Art. 6.1 LP 11/1986)

Claims Claims 1-3 IF NOT

Inventive activity (Art. 8.1 LP11 / 1986)

Claims Claims 1-3 IF NOT

The application is considered to comply with the industrial application requirement. This requirement was evaluated during the formal and technical examination phase of the application (Article 31.2 Law 11/1986).  Opinion Base.- This opinion has been made on the basis of the patent application as published.  WRITTEN OPINION 1. Documents considered.- The documents belonging to the state of the art taken into consideration for the realization of this opinion are listed below.

Document

Publication or Identification Number publication date

D01

US 4926346 A (YOKOYAMA SHOUJI) 15.05.1990

D02

ES 2286973 T3 (BAYERISCHE MOTOREN WERKE AG) 16.12.2007

D03

DE 102004051527 A1 (DAIMLER CHRYSLER AG) 06.06.2006

D04

US 5559695 A (DAILY MICHAEL J) 24.09.1996

2. Statement motivated according to articles 29.6 and 29.7 of the Regulations for the execution of Law 11/1986, of March 20, on Patents on novelty and inventive activity; quotes and explanations in support of this statement Document D1 can be considered as the representative of the closest state of the art since most of the claimed technical characteristics converge in this document. Contrast of the application with document D1 Independent claims: Claim 1 Image acquisition system for driving assistance applications of automatically adaptable motor vehicles according to the driving environment and the vehicle itself, comprising: -a camera (1) of variable focal optics installed in the vehicle responsible for capturing the images (see this in document 1 in column 2, lines 54-68; column 3, lines 58-68, see reference 1 and figure 2); -a control module (2) of the camera (1) configured to receive from each driving assistance application (3) a status information indicating whether the application (3) is active or not and an optimum operating range of the viewing angle of the camera (1) and, depending on this data, adjust the viewing angle of the camera (1) to optimize the operation of the vision applications (3) that are active (read in document 1 reference 10 (CPU) and read column 3, lines 58-68; column 4, lines 1-2, see also figure 2, figure 4 and figure 5a and claims 1.3-5). Therefore, claim 1 is not new (Art. 6.1 LP 11/1986) as it is affected by D1 Dependent claims: Claims 2 Image acquisition system where the control module (2) is additionally configured to receive information from a plurality of sensors (4) installed in the vehicle and use said information, together with that supplied by the driving assistance applications (3), to determine the camera's angle of view adjustment (1) (this is observed in the document D1 in Figure 2; see column 3, lines 58-68; column 4, lines 1-24 and read claim 2). Claim 2, therefore, is not new (Art. 6.1 LP 11/1986). Claim 3 Image acquisition system according to any of the preceding claims, wherein the driving assistance applications (3) are additionally configured to receive information from a plurality of sensors (4) installed in the vehicle and use said information to define at least one of the following: -the optimum operating range of camera viewing angle (1); -the state of the driving assistance application (3). (See column 3, lines 1-7, read claims 1.3-5). Claim 3, therefore, is not new (Art. 6.1 LP 11/1986). Contrast of the application with the document D2 Independent claims: Claim 1 Image acquisition system for driving assistance applications of motor vehicles automatically adaptable according to the driving environment and the vehicle itself, comprising: -a camera (1) of variable focal optics installed in the vehicle in charge of capturing the images; -a control module (2) of the camera (1) configured to receive from each driving assistance application (3) a status information indicating whether the application (3) is active or not and an optimum operating range of the viewing angle of the camera (1) and, depending on these data, adjust the viewing angle of the camera (1) to optimize the operation of the vision applications (3) that are active. (In claim 1 of document D2 are all the elements of claim 1 of the patent application. See also Figures 1, 2a and 2b. The chamber is reference 10). Therefore claim 1 is not new (Art. 6.1 LP 11/1986) as it is affected by D1  WRITTEN OPINION Dependent claims: Claims 2 Image acquisition system where the control module (2) is additionally configured to receive information of a plurality of sensors (4) installed in the vehicle and use said information, together with that supplied by the driving assistance applications (3), to determine the camera's angle of view adjustment (1). (In the claim 2 are the elements of claim 2 of the patent application, also read column 2, lines 3436). Claim 2, therefore, is not new (Art. 6.1 LP 11/1986). Claim 3 Image acquisition system according to any of the preceding claims, wherein the applications of driving assistance (3) are additionally configured to receive information from a plurality of sensors (4) installed in the vehicle and use this information to define at least one of the following: -the optimum operating range of camera viewing angle (1); -the state of the driving assistance application (3). (Note in claims 1-3, there are the elements of claim 3 of the patent application). Claim 3, therefore, is not new (Art. 6.1 LP 11/1986).