JP4967758B2 - Object movement detection method and detection apparatus - Google Patents

Description

  The present invention relates to object movement detection and position detection based on an image captured by an image acquisition device.

  Real-time detection of the movement of an object in a landscape (three-dimensional space region) and detection of its position by performing processing of a captured image are areas where demand is increasing. In particular, the image acquisition device is movable with respect to the landscape (for example, it can be operated with visible or invisible lines such as infrared rays). For example, the movement of parts is monitored and obstacles within the movement range of the robot It is widely applied to areas related to vehicle safety using video surveillance in critical areas (such as intrusion detection, especially related to security), collision possibility detection system, and driver assistance system (such as parking assistance). The

  Particularly in the field of vehicle safety, there is an increasing need to acquire and process images. The present invention is not limited to this field, but may be applied to other fields, particularly those described above. In the following, the field mainly related to vehicle safety will be considered.

  An example of the field related to vehicle safety is the parking assistance system (PAS) of Non-Patent Document 1 developed by IMRA-Europe. This system is a system that makes it possible to acquire a plurality of images of different positions of a vehicle with respect to a landscape by an in-vehicle CCD camera, and to reproduce the surrounding landscape of the vehicle as a 3D image based on the images. However, in this system, the movement of the vehicle is necessary, the object in the landscape must be stationary, and interference is likely to occur when the object moves with respect to the landscape.

  Detecting the movement of an object from a movable image acquisition device is an area that requires a great deal of research activity, and the major problem is the actual movement or between the image acquisition device and the acquired image. It is difficult to distinguish whether the movement is due to relative movement.

  Known methods for detecting the movement of an object in a landscape include those having a first method group and a second method group. The first method group detects the moving area in the landscape and then integrates the moving area to remove noise. This is done by two means.

  As a first means, it is applied to a video surveillance system using a fixed camera, and the difference between a plurality of images acquired by the fixed camera, the difference between the current image and the previous image, or the current image This is based on the difference between an image and a background image (an image accumulated over time). However, although this first means is common in many applications, when the camera moving with respect to the landscape captures a very close landscape, accurate processing is not performed.

  As other means, there is an optical flow clustering method. This method distinguishes between the actual movement of an object in the landscape and the movement of the object that occurs simply because the camera moves relative to the landscape. Specifically, for example, the method described in Non-Patent Document 2 can be given. These methods allow processing when the camera is moving, and the object movement relative to the landscape, calculated by the difference based on the optical flow caused by the apparent object movement caused by the camera movement. The object is a stationary object with respect to the landscape (eg the ground) that has an optical flow pointing in a clearly defined direction associated with the movement of the camera relative to the landscape. , Objects that move relative to the landscape, each with an optical flow pointing in a different direction. However, these optical flow clustering methods have a drawback that a large amount of calculation time is required to match a large number of images.

  The second group of methods for detecting the movement of objects in the landscape is based on detecting the movement of specific objects (eg, cars and pedestrians). These methods are performed based on shape recognition, and are used to track the movement of a specific object in a landscape (Patent Document 1). As a new trend related to object detection, there is an introduction of a method that focuses on searching for a feature of a specific object. For example, there is one that recognizes walking of a person (Patent Document 2). Furthermore, vertical edges and symmetry (Non-Patent Document 3), and a difference in texture or property developed based on a genetic algorithm (Non-Patent Document 4) can be mentioned. The properties obtained in this way are integrated by a neutral network or a support vector machine (SVM).

  When the moving object (or shape) is not grasped in advance, or when the image of the projected object is captured at a short distance, and the image is distorted as if it was captured with a wide-angle lens camera For example, the above method is not appropriate.

  Some known methods detect the movement of an object using inverse perspective mapping (IPM) (Non-Patent Document 5). In this method, the image converted in this way is analyzed together with the optical flow cluster method to first remove the perspective effect, and the landscape image is converted to detect movement. However, these methods have the drawbacks described above when optical flow clustering is performed, and thus apply to the method described in the following article, for example (Non-patent Document 6).

Further, as in the invention disclosed in Patent Document 3, in order to recognize the movement of an object, it is disadvantageous that two cameras are necessary to capture one landscape from two directions at the same time. There is.
International Publication No. 2005/098739 Pamphlet International Publication No. 2005/050565 Pamphlet European Patent No. 360880 K. Finzel, R. Bendahan, C. Vestri, S. Bougnoux and T. Kakinami, “3D Parking Assistant System”, 11th World Congress on Intelligent Transport Systems, Nagoya, Japan, 2004 G.Lefaix, E. Marchand, P.Bouthemy, “Motion-based Obstacle Detection and Tracking for Car Driving Assistance”, International Pattern Recognition Federation Bulletin, ICPR 2002, Vol. 4, p. 74-77, Quebec, Canada, August 2002 A.Broggi et al., “A Multi-Resolution Approach for Infrared Vision-Based Pedestrian Detection”, IEEE Intelligent Vehicle Symposium, p.7-12, 2004 (See Y. Abramson, B. Steux, “Hardware-friendly pedestrian detection and impact prediction”, Proceedings of IEEE Intelligent Vehicle Symposium, June 2004) (WMNewman, RFSproul, “Principles of Interactive Computer Graphics”, McGraw-Hill, Tokyo, 1981) HAMallot, HHButhoff, JJLittle, S. Bohrer: “Inverse perspective mapping simplifies optical flow computation and obstacle detection”, Biological Cybernetics 64, p. 177-185, 1991.

  The present invention solves the above problems, and does not require knowledge of the type of object (shape or other properties) in advance, confirmation of the object, enormous calculation time or large calculation means, and the like. An object movement detection method and a detection apparatus capable of processing an image and detecting movement of an object in a landscape are provided.

(1) the reference plane based on the processing by the arithmetic unit of the plurality of scene images captured by the movable camera (1) with respect to the detection of an object moving in the scene (6) comprising a portion of the reference surface in the method, the reference surface data for some shape and position of the reference plane is stored in the memory of the computing device, the posture of the camera, each image of scenery that has been transmitted to the imaged the arithmetic unit by the camera stored in the memory of the calibration parameters determined for said computing device, the computing device can receive the still object data relating to an object that is stationary relative to a part of the reference plane detected in the landscape in stores the still object data in the memory, in the detection method of the object movement can output the stored data,
Executed by the arithmetic unit,
For some of the reference plane, the reference plane data stored in said memory, corresponding pose of the camera, using the calibration parameters, captured by the camera, it is transmitted to the computing device, landscape Performing reverse perspective mapping to determine a plurality of corresponding transformed images in a plurality of consecutive images (21) of:
For each detected stationary object (10), on the basis and the stationary object data stored in memory corresponding pose parameters of the camera, converting the image corresponding masked with respect to the camera by the stationary object Determining an area (12), determining a background map (25) based on a change in color intensity between corresponding pixels in the converted image obtained in step (a), and masking the converted image area Taking into account this determined masked area (24) and a part of the background map, a movement map (26) that is constituted by pixels and determines whether it is related to the presence of object movement An object movement detection method comprising the step (a ′) of calculating .

(2) the step (a ') further the likelihood value which the mobile is performed, in association with the step (a) the pixels in said transformed image selected from the transformed image obtained at the mobile establishing the likelihood map, for that pixel has a color intensity value corresponding pixels hardly vary image in the selected transmission image from among the obtained at step (a) image, the camera considering each area masked against movement expected that are associated with the pixel, the higher the color intensity value of a corresponding pixel of said step (a) at a final said transformed image of obtained were selected the different from the color intensity values of the corresponding pixels in the converted image, each pixel is determined to be related to movement depending on the likelihood of the associated mobile Detection method for object movement according to claim 1, characterized in that it comprises a step corresponding to a pixel of said mobile likelihood map that.

(3) the step (a ') is further said arithmetic unit is performed by, whenever the color intensity difference between the corresponding pixel in the selected said converted image is sufficiently small, the background map within corresponding thereto the pixel, the by assigning a valid color intensity value close to the average color intensity value of the corresponding pixel, a corresponding at least two said transformed image selected from a plurality transformed image obtained in step (a) determining the background map by calculating the color intensity difference between pixels, and step (b) constituting the said each pixel valid color intensity values assigned colored pixels, the calculated in step (a) was, among the transformed image a plurality of continuous, performs area masking operations on the end of said transformed image (22), determined al And storing the area masking data in said memory (c), a plurality of said converted image obtained in the step (a), are determined in the last of said transformed image, said step (b) It said background map, the step (c) in the obtained corresponding area masking data, the mobile likelihood value based on said mobile likelihood map corresponding is associated with each pixel for colored pixels in the background map, the background the corresponding colored pixels in the map, the pixel difference is large in color intensity between the corresponding pixels in the final image, assigning a high mobility likelihood, in the background map and the corresponding colored pixels, of the final Assign a low likelihood to a pixel that has a small color intensity difference between corresponding pixels in the image, and If the pixel is located in an area masked, and storing this manner reduces the likelihood of a mobile assigned to pixels, the movement likelihood maps obtained in the memory (d), the step in said mobile likelihood maps obtained by (d), range associated within each pixel, by determining whether associated with sufficiently high mobility expected, to filter the mobile prospective map, and step (e) for storing the filtering after the movement likelihood maps obtained in the memory, for determining the moving map, for each relevant range of the filtering process after the movement likelihood map that is determined in the previous step, (F) having a step (f) of determining whether the relevant range coincides with the presence of movement of the object based on a predetermined moving direction criterion; Movement detection method of the object movement according to claim 1 or 2, characterized in Rukoto.

(4) the by the arithmetic unit, for each relevant range of the moving map the obtained in step (e), to determine the closest point of the projected optical center, the camera on the reference plane the basis of the calculation of the position of the projected optical center C, has a step (g) to determine the position of the moving object in the level of the reference surface on the moving map obtained in the step (f), detection method for object movement according to claim 3 this way each point is determined by the, characterized in that corresponding to the contact point of the moving object and the reference plane.

(5) based on the moving object detection method further plurality of successive said mobile maps obtained with the lapse of time obtained in said step (g), a succession of representative points corresponding with the computing device the position calculated monitoring, comprising the steps (h) determining the expected position of these representative points, each of these representative points, claim 4 corresponding to the contact point of the moving object on the reference plane The object movement detection method described in 1.

(6) Detection method of the object movement, the following additional steps performed using the computing device, i.e. from the last image of the plurality of said converted image obtained in the step (a), on the basis of the stored the reference surface data into the memory, a detection step of detecting a pixel corresponding to a portion of the reference surface, corresponding to the detected pixel zero change estimated as corresponding to the reference plane detection method for object movement according to any one of claims 2 to 5 having an assignment step of the allocation to each pixel in said mobile likelihood map to.

(7) the detecting step the object movement detection method further the executed by the computing device, a final said conversion image previously sent image of, for detecting the pixel corresponding to a portion of the reference surface When, the high mobility expected, and having an allocation step of allocating to each pixel in said detected movement expected map as belonging to the reference plane in a previous said transformed image having a non-zero movement prospect The object movement detection method according to claim 6.

In the apparatus for detecting a movement of an object in scene (6) having a portion (8) a reference plane, movable with respect to the reference surface, the position of the fundamental spatial reference frame associated with the reference surface A mount (2) comprising a position sensor (4) capable of transmitting data to be defined;
Attached to said mount, landscape (6) is capable of capturing an image, and a predetermined posture parameters for the mount, the camera (1) capable of transmitting an imaging image with predetermined intraosseously calibration parameters, of the reference plane the reference surface data relating to a part of the shape location, a mounting position data transmitted by the position sensor, the image data transmitted by the camera, and camera posture parameter data relating to the mount and the camera calibration parameters, external capable of receiving computing device stationary object data relating to the detected stationary object in the scene transmitted from the environment and a (5), wherein the computing device is capable calculating the orientation parameters of the basic camera reference frame Received image data can be processed and received Have and can store calculation data memory, the detection device can transmit the stored data, said arithmetic unit, to obtain a plurality corresponding transformed image relates portion of the reference surface, stored in the memory criteria plane data, the attitude of the corresponding camera, by using a calibration parameter, and execute the inverse perspective mapping landscape image (21) to a plurality of successive received from the camera (a), the detected still to object (10), on the basis a memory stored the still object data to the corresponding camera pose parameters, and determine the corresponding conversion image area is masked (12) relative to the camera by the stationary object, this run the step of masking the areas (a '), the color between corresponding pixels of the transformed image that is stopped even step (a) Based on the calculation of the change in the degree, considering each mask area determined, calculated each pixel, moving map constituted by the associated pixel to the movement of the object relative to the portion of the reference surface is present (26) and, detecting device object movement, characterized in that it comprises storing said moving map in the memory.

(9) the computing device further the likelihood value movement is taking place, the movement likelihood map by an association step (a) the pixels in said transformed image selected from the transformed image obtained in establishing, with respect to the pixel has a color intensity value corresponding pixels hardly vary image in the transmission image that was selected from among the obtained at step (a) image, with respect to the camera considering each area masked, move expected that are associated with the pixel, the higher the color intensity value of a corresponding pixel in the transformed image of the last of the obtained in step (a) is the transformed image selected different from the color intensity value of a corresponding pixel in the inner, the each pixel is determined to be related to movement depending on the likelihood of the associated mobile Detector of object movement according to claim 8, characterized in that it comprises a step corresponding to a pixel of the dynamic likelihood map.

(10) the computing device further whenever the color intensity difference between the corresponding pixel in the selected the transformed image is sufficiently small, the pixels in the background map corresponding to their average color intensity of the corresponding pixel by assigning a valid color intensity value close to the value to calculate the color intensity difference between corresponding pixels of at least two said transformed image selected from the plurality the transformed image obtained in step (a) determining the background map by a step (b) constituting the respective valid color intensity value is assigned pixel color pixel, the obtained in step (a), of the converted image in which a plurality of successive the out performs area masking operations on the end of said transformed image (22), wherein the area masking data obtained notes And step (c) to be stored in said step in a plurality of said converted image obtained (a), the last of the transformed image, the background maps obtained in the step (b), the step ( corresponding area masking data obtained in c), based on the movement likelihood map mobile likelihood value corresponding is associated with each pixel for colored pixels in the background map, the corresponding colored pixels in the background map, the the final pixel difference is large in color intensity between the corresponding pixels in the image, assigning a high mobility likelihood, corresponds a colored pixels, color intensity between the corresponding pixels in the final image in the background map If a pixel with a small difference is assigned a low likelihood of movement, and the pixel is located in the masked area In this way, to reduce the likelihood of a mobile assigned to pixels, and the step (d) storing the mobile likelihood maps obtained in the memory, the mobile likelihood maps obtained in step (d) at, range associated within each pixel, by determining whether associated with sufficiently high mobility expected, performs the movement likelihood map to filter, the filtering process after the movement likelihood maps obtained above and step (e) to be stored in memory, to determine the moving map, for each relevant range of the filtering process after the movement likelihood map that is determined in the previous step, based on a predetermined moving direction reference, the relevant a moving object according to claim 8, range and wherein the performing step (f) to determine whether consistent with the presence of movement of the object Detection device.

  Moreover, what solves the said subject is as follows.

  The computing device further determines the projected optical center of the camera on the reference plane by determining the closest point of the projected optical center for each relevant range of the movement map determined in step (e). Based on the calculation of the position of C, step (g) for determining the position of the moving object at the level of the reference plane on the movement map obtained in step (f) is executed, and each point thus determined is The object movement detection device according to claim 1, wherein the object position data is stored corresponding to a contact point between the moving object and the reference plane.

  The computing device further calculates and monitors the continuous positions of the corresponding representative points using the computing device based on the plurality of continuous movement maps obtained with the passage of time in step (g). The step (h) of determining the predicted position of the representative point is executed, and each of the representative points is preferably the object movement detection device corresponding to the contact point of the moving object on the reference plane.

  Further, the arithmetic unit further corresponds to a part of the reference plane based on the reference plane data stored in the memory from the last image among the plurality of converted images obtained in step (a). Assigns each pixel in the moving likelihood map corresponding to the pixel detected as corresponding to the reference plane, and stores it in the modified moving map memory. It is preferable that the object movement detection device described above be used.

  In addition, the arithmetic unit further includes a detection step for detecting pixels corresponding to a part of the reference plane in the image transmitted before the final conversion image and a high movement probability, and a previous conversion image having a non-zero movement probability. It is preferable that the object movement detection device execute an assigning step of assigning to each pixel in the expected movement map detected as belonging to the reference plane.

  Further, the movement detection device is capable of detecting a stationary object in a landscape and cooperates with the stationary object detection device capable of transmitting data related to the detection of the stationary object to the arithmetic device. An apparatus is preferred.

  The object movement detection device is preferably the object movement detection device described above, which is capable of transmitting movement map data to an operator.

  In addition, the object movement detection device can cooperate with the collision prevention warning device, and can calculate data related to the risk of collision with an object in the landscape based on the movement map data. It is preferable that the object movement detection device is capable of transmitting data to a collision prevention warning device.

  In addition, the object movement detection device cooperates with the collision prevention warning device, and calculates data related to the risk of collision with an object in the landscape based on the determination data that defines the predicted position of the representative point by the arithmetic device. In order to predict the risk of collision, it is preferable that the object movement detection device is capable of transmitting data related to the collision risk to the collision prevention warning device.

  Further, the collision prevention warning device is preferably the object movement detection device characterized in that information on a risk of collision with an object in a landscape can be transmitted to an operator.

  Further, the arithmetic device can transmit the image image to a screen capable of displaying the received image, and the arithmetic device can display display data regarding the risk of collision with the object in the landscape based on the calculation of the risk of collision with the object. Preferably, the object movement detection device is capable of calculating, transmitting the risk display data to a screen, and displaying the display data cited by the screen as an image.

  Further, the object movement detection device is connected to a correction device capable of correcting the movement of the mount relative to the reference plane based on data transmitted by the device that detects the movement. A detection device is preferred.

  In addition, the vehicle that can move on the ground provided with a device for detecting object movement is the object movement detection device, wherein the vehicle constitutes a camera and the ground around the vehicle constitutes a reference plane. Preferably there is.

The above configuration will be further described. The present invention relates to an object in a landscape (6) including a part of a reference plane, based on processing by a computing device of a plurality of landscape image images captured by a calibration camera (1) movable with respect to the reference plane. In the movement detection method, the reference plane data relating to the shape and position of a part of the reference plane is stored in the memory of the arithmetic unit, and for the posture of the camera and each image of the landscape imaged by the camera and transmitted to the arithmetic unit The determined calibration parameters are stored in the memory of the arithmetic unit, and the arithmetic unit can receive stationary object data related to an object that is stationary with respect to a part of the reference plane detected in the landscape. Stored in the memory with respect to a part of the reference plane, which is executed by the arithmetic unit in a movement detection method capable of outputting the stored data In order to obtain a plurality of corresponding converted images in a plurality of continuous images (21) in a landscape that are imaged by the camera and transmitted to the arithmetic unit using the data, the corresponding camera posture, and the calibration parameters. , Performing the reverse perspective mapping, and for each detected stationary object (10), based on the stationary object data stored in the memory and the corresponding camera posture parameters, the stationary object A corresponding transformed image area (12) to be masked against and a reference plane (25) based on the change in color intensity between the corresponding pixels in the transformed image determined in step (a) is determined and transformed A step of masking the image area (12) is performed, and the determined masked area (24) and the reference plane (25) Considering part is constituted by the pixel, but can be realized by the movement detection method comprising the step (a ') for calculating a moving map that determines whether associated with the presence of an object moving in (26).

  In fact, according to the movement detection method of the present invention, the basic property of inverse perspective mapping (IPM) of a landscape image taken on the image plane of the camera is used, and along a straight line passing through the optical center of the camera. Thus, it is possible to considerably simplify the detection operation by projecting each point on the image plane to correspond to the reference plane. That is, it automatically corrects the movement of the camera relative to the reference plane of the landscape (see H.A.Mallot literature paragraph 3.3, pages 180-181 above for this basic property). According to the present invention, it is possible to use the basic property of IPM conversion by an operation that can be easily performed by analyzing the change in color intensity between corresponding pixels of an image image after multiple conversion and image processing. Although a method for analyzing a change in color intensity is known, this movement detection method has been applied directly to a captured image rather than to a converted image.

  Furthermore, area masking by locating at the camera viewpoint (field of view concentrated on the optical axis of the camera) enables analysis of the result of object detection stationary with respect to the reference plane (eg IMRA European patent FR 2853121). The result obtained by using a stationary object detector, a laser scanner, or a pre-established mapping of an object existing in the vicinity). This eliminates the need to consider the image pixels corresponding to the mask area, simplifies the calculation, and further ensures the calculation of the movement map. Therefore, the present invention can be used with a simple on-board computer that is currently widely used when used in a vehicle parking device (such as a vehicle equipped with a camera for monitoring the rear of the vehicle when reversing). is there.

  As one method of the movement detection method according to the present invention, the above step (a ′) further includes calculating a likelihood that the movement is being performed, by calculating a pixel in the conversion image selected from the conversion image obtained in step (a). Establishing a moving likelihood map by associating the image with a color intensity value for which the corresponding pixel in the transmitted image selected from the images obtained in step (a) hardly changes from image to image. Considering each area masked to the camera, the likelihood of movement associated with the pixel is selected if the color intensity value of the corresponding pixel in the final transformed image determined in step (a) is high Different from the color intensity values of the corresponding pixels in the converted image, each pixel is determined to be associated with a movement that depends on the likelihood of the associated movement. Those having a step corresponding to the pixel in the moving prospective map that.

  Even if it is used as it is with an existing relatively weak computer by this prospect data processing, the accuracy of the result can be improved.

  A further method of the movement detection method according to the present invention described in any of the above embodiments is based on one of the above-described forms, and by considering the change in color intensity and filtering the prospect map, Step (a ') is further performed whenever the color intensity difference between the corresponding pixels in the selected transformed image is small enough to be performed by the arithmetic unit. By assigning the effective color intensity value close to the average color intensity value of the corresponding pixel to the pixels in the background map, the pixels in at least two converted images selected from the plurality of converted images obtained in step (a) The background map (25) is determined by calculating the color intensity difference between the corresponding pixels, and the effective color intensity value is assigned. Each pixel is obtained by performing an area masking operation on the last converted image (22) of a plurality of consecutive converted images obtained in step (b) and step (a) constituting a colored pixel. (C) storing the area masking data in the memory, the last converted image in the plurality of converted images obtained in step (a), the background map obtained in step (b), the corresponding area masking data determined in c), the corresponding color pixel in the background map based on the corresponding motion likelihood map associated with each pixel for the color pixel in the background map. Assign a high likelihood to a pixel that has a large color intensity difference between corresponding pixels in the image, Assign a low likelihood of movement to a pixel with a small color intensity difference between the corresponding colored pixel in the background map and the corresponding pixel in the final image, and the pixel is located in the masked area In the step (d) of reducing the likelihood of movement assigned to the pixels in this way and storing the determined movement probability map in the memory, and in the movement probability map determined in step (d), each pixel Determining whether or not the related range in is associated with a sufficiently high likelihood of moving, filtering the predicted movement map, and storing the obtained filtered predicted movement map in memory ( e) and each related range of the post-filtering estimated movement map determined in the previous step to obtain the movement map. And, based on the predetermined moving direction reference, the relevant range is one having a step (f) to determine whether consistent with the presence of the moving object.

  Furthermore, the present invention can accurately indicate the position of an object from which movement has been detected. The movement detection apparatus according to the present invention based on the above-described form is configured such that the calculation device determines the closest point of the projected optical center for each related range of the movement map obtained in step (e). Determining the position of the moving object at the level of the reference plane on the movement map obtained in step (f) based on the calculation of the position of the projected optical center C of the camera on the surface; Each point determined in this way corresponds to a contact point between the moving object and the reference plane.

  Based on the above embodiment, the movement detection method further includes successive positions of corresponding representative points using a computing device based on a plurality of continuous movement maps obtained with the passage of time obtained in step (g). The step (h) of calculating and monitoring and determining the expected positions of these representative points each corresponds to the contact point of the moving object on the reference plane. The predicted positions of these representative points may be actually subtracted directly from the trajectory of the stored points (based on the passage of time between successive positions and the calculated speed and acceleration). For example, Kalman filtering can be used to calculate these predicted positions. This form of the invention has the advantage of steadily detecting the direction of movement, which can be caused by parasitic movements such as detection errors, sudden changes in lighting conditions, shadow movements on the reference plane, etc. This is possible because it can be filtered out.

  As another form of the movement detection method according to the present invention, the reliability of the movement prospect map is improved based on the above-described form, and the movement detection method further includes the following steps performed by using the arithmetic device, that is, A detecting step for detecting a pixel corresponding to a part of the reference plane based on the reference plane data stored in the memory from among the last images among the plurality of converted images obtained in step (a); Assigning a zero change probability to each pixel in the movement likelihood map corresponding to the pixel detected as corresponding to the reference plane.

In the present invention, it is possible to further improve the reliability of the movement likelihood map by considering the difference in movement likelihood associated with the pixel between the latest conversion image and the previous conversion image. As another aspect of the present invention based on the above aspect, the movement detection method is further executed by an arithmetic device.
A detection step for detecting pixels corresponding to a part of the reference plane in the image transmitted before the final conversion image, and a high movement probability, which has a non-zero movement probability and belongs to the reference plane in the previous conversion image And assigning to each pixel in the movement prospect map detected as a thing.

Further, an object movement detection apparatus capable of using the movement detection method according to the present invention, particularly an apparatus for detecting the movement of an object in a landscape (6) having a part of a reference plane,
A mount having a position sensor (4) that is movable relative to the reference plane and capable of transmitting data defining a position in a basic spatial reference frame associated with the reference plane; (6) A camera (1) capable of capturing an image and capable of transmitting a captured image having a predetermined attitude parameter with respect to the mount and a predetermined internal calibration parameter, and a reference plane relating to the shape and position of a part of the reference plane Data, mount position data transmitted by the position sensor (4), image data transmitted by the camera, camera posture parameter data regarding the mount and internal camera calibration parameters, and detection in the landscape transmitted from the external environment And an arithmetic unit (5) capable of receiving stationary object data related to the stationary object In the detection device capable of calculating the camera attitude parameters related to the basic reference frame, processing the received image data, storing the received / calculated data, and transmitting the stored data In order to obtain a plurality of conversion images, the arithmetic unit uses a plurality of continuous images received from the camera using the reference plane data stored in the memory, the corresponding camera attitude, and the calibration parameters for a part of the reference plane. The reverse perspective mapping of the landscape image (21) to be executed is performed (a), and each detected stationary object (10) is determined by the stationary object based on the stationary object data stored in the memory and the corresponding camera posture parameter. The corresponding conversion image area (12) to be masked with respect to the camera is determined, and the process of masking this area is executed ( a '), based on the calculation of the change in color intensity between the corresponding pixels of the converted image obtained in step (a), taking into account each determined mask area, each pixel moves the object relative to the reference plane part Calculating a movement map (26) constituted by the pixels associated with the presence of and storing the movement map in memory.

  Further, the arithmetic unit of the movement detecting device further associates the estimated value of the movement with the pixel in the converted image selected from the converted image obtained in the step (a), and displays the estimated movement map. For that pixel, the corresponding pixel in the transmitted image selected from the images determined in step (a) has a color intensity value that varies little from image to image and is masked to the camera. Taking into account each area, the likelihood of movement associated with a pixel is determined if the color intensity value of the corresponding pixel in the final converted image obtained in step (a) is high. Unlike color intensity values, each pixel is a pixel in the movement likelihood map that is determined to be associated with movement that depends on the associated movement likelihood. And it has a step corresponding to Le.

  Various movement detection devices according to the present invention will be described below. In particular, an apparatus capable of indicating the position of an object where movement has been detected will be described in detail.

  In the present invention, the movement detection device can detect a stationary object in a landscape and can cooperate with a stationary object detection device capable of transmitting data related to the detection of the stationary object to the arithmetic device.

  Since the movement detection device can transmit movement map data to the operator, for example, when there is a risk that the camera mount will collide with an object, the operator can react to the transmitted data. is there. The movement detection device cooperates with the collision prevention warning device, and can calculate data related to the risk of collision with an object in the landscape based on the movement map data by the arithmetic device. It can be transmitted to a warning device and may provide automatic detection of a collision. The collision prevention warning device may transmit information related to the risk of collision with an object in the landscape to the operator. In addition, in order to predict the risk of collision, it is possible to calculate data related to the risk of collision with an object in the landscape based on the determination data that defines the predicted position of the representative point (see above). The data regarding the risk of collision may be transmitted to the collision prevention warning device. The risk of collision may be calculated based on movement map data or determination data on the expected position of a representative point.

  In the movement detection apparatus according to any of the embodiments, the movement detection apparatus may be connected to a correction apparatus that can correct the movement of the mount other than the reference surface based on the data transmitted by the movement detection apparatus. In this case, obstacle avoidance or mount steering operation may be performed automatically or with assistance from the operator. Such a device is particularly useful in the field of automotive safety (in this case the mount is a vehicle) or in the field of robotic optics (in this case the mount is a robot or an operable part of the robot). ).

  Finally, the present invention relates to a vehicle that can move on the ground, and includes a device that detects movement at any location around the vehicle. Therefore, the vehicle constitutes a camera mount, and the ground around the vehicle constitutes a reference plane.

  According to the detection method and the detection apparatus of the present invention, it is not necessary to acquire knowledge of the type of object (shape or other properties) in advance, confirm the object, enormous calculation time or large calculation means, and the like. It is possible to process and detect the movement of an object in the landscape.

  An embodiment of the present invention relating to an in-vehicle movement detection device related to a device used for vehicle parking assistance will be described with reference to FIGS.

  At the rear of the vehicle, a CCD camera (1) is mounted at a predetermined location where a landscape image from the ground to the rear of the vehicle or a 3D monitoring area can be imaged. The ground has a reference plane S on which the vehicle can move. Here, in the vehicle (2), the position sensor (4) is arranged on the wheel (ABS sensor) or on the on-board computer (5) (having the arithmetic unit and the memory). Each sensor (4) is connected to an on-board computer (5), can detect the position of the vehicle with respect to the ground, and outputs acquired position data to the computer (5). Of course, the position of the vehicle is not limited to this as long as the position of the vehicle can be measured. For example, the position of the vehicle may be measured using a satellite-based positioning system (GPS system).

  The camera (1) can output a captured image to an arithmetic device of the on-board computer (5), and the input image can be processed by the arithmetic device of the computer. The camera characteristic parameters are camera posture and calibration parameters.The camera posture parameter indicates the coordinates of the optical center C in the spatial reference frame R (x, y, z axes) linked to the ground, for example. The path of the camera when the vehicle moves is determined. This is because it is necessary to know the position of the camera with respect to the reference plane of each captured image (more specifically, the optical center point C) in order to calculate reverse perspective mapping. The posture parameter is mainly included in the incidental calibration parameter of the camera. For example, the posture parameter of the camera (1) can be set in the reference frame R by simply changing the reference frame based on the position of the camera found in the coordinate system centered on the vehicle (for example, the position of the optical center). Obviously, it is subtracted from the data defining the position of the vehicle at. The computer (5) can process the position data received from the sensor (4), and from this data, the attitude of the camera (1) can be cross-referenced to the corresponding camera attitude from the image received from the camera. The parameter can be estimated.

  The camera (1) in this case is a slit-type CCD camera (see F. Duvernay and O. Faugeras, Machine Vision and Applications (13), 14-24 (2001)), which has been distorted. is there. In the slit model camera, all straight lines in the space on the image plane are imaged as straight lines.

  In addition to the distortion parameters, there are the following 10 camera calibration parameters and attitude parameters.

4 internal parameters
a: Horizontal focal length of the camera
b: Vertical focal length of the camera
c: Horizontal coordinate of optical center
d: The vertical coordinate “a” of the optical center represents the product of the pixel length and the focal length f of the camera, and “b” represents the product of the pixel width and the focal length f of the camera.

  The coordinates c and d of the frame R '(u, v axes) on the image plane (camera imaging plane at a distance f from the optical center) represent the projection of the optical center C along the optical axis in the image plane. .

Traditionally, these four parameters constitute the sequence П (3 × 4) (Equation 1 below).

-Three external parameters (posture parameters) that make up the vector t representing the position of the optical center of the camera in the frame R
-Rotation column r (3x3 column) representing three rotations linked to each of the optical axis tilt angle, the rotation angle along the optical axis, and the yaw angle, three external parameters (r is three independent parameters) Rotation product)
In the conventional form, the four internal parameters (other than the strain parameter) are grouped as a column П, and the six external parameters are grouped as a column K (4x4,) expressed as .

  If the coordinates of any point Pi in the landscape (6) are measured in R and described in the form of homogeneous coordinates, Pi = [pix, piy, piz, 1] T.

  And if Ci is the corresponding point in the captured image, once this point Ci has been corrected for distortion effects (any of the well-known methods may be used, eg F.Devernay and O. above). (See the Faugeras reference), the point Ci yields the point Di: Di = F (Ci), where F represents the distortion function using the distortion parameters.

  When it cannot be limited to the slit camera, there are other distortion models and calibration models for these distortion parameters. For example, the document “Camera calibration with distortion models and accuracy evaluation” by J. Weng, P. Cohen, M. Herniou, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14 (10), p. 965-980, October 1992. It is done.

  The homogeneous coordinates of the point Ci in R ′ are usually expressed as [u, v, 1] (the index T represents the dislocation), and the coordinates of the correction point Di are ki. [Ui, vi, 1] T Shown in shape. Here, ki represents the depth of the optical axis between the corresponding predetermined point Pi and point Di (ki.ui and ki.vi are coordinates after distortion correction).

  The equation resulting from a one-to-one match between a given point on the scene and a point in the image where the distortion effect has been corrected is expressed as Di = П.K.Pi. By reversing this relationship, one predetermined point Pi can be moved to the corresponding predetermined point Di.

  The computing device of the computer (5) can easily correct the influence of distortion in the image taken by the camera.

  After correcting for the effects of distortion, the computing device can perform reverse perspective mapping on the points in the image taken by the camera. Inverse perspective mapping is well known in the field of image processing, and is to associate a reference plane point D ', in this case the ground, with a point (or pixel) D on the image plane P of the camera. . Here, the point D ′ represents a point (or pixel) where a straight line (optical ray) connecting the optical center C of the camera and the image plane P intersects (with the reference plane S). As shown in FIG. 2, the optical center C is separated from the image plane P by a distance f (along the optical axis). The position of the image plane in the space can be known from the stored camera posture and calibration data, and the position of the reference plane in the space from the stored reference plane data regarding the shape and position. Then, the converted image obtained by projecting onto the reference plane along the optical ray (passing through the optical center) is matched with the image captured by the camera. The result of reverse perspective mapping is represented in the form of clusters of colored pixels in the ground image that make up the transformed image. In addition, interpolation can be performed so as to fill a space appearing between clusters, and overlapping clusters can be integrated.

  Usually, for simplicity of calculation, the ground is considered to be flat as shown in FIG. However, even when the ground is not flat, it is sufficient to calculate the point of the optical ray that extends from the image plane and intersects the reference plane (ground), and the predetermined (stored in memory) position and shape parameters. By configuring the reference plane using the image captured by the camera, it can be easily processed. Thereafter, movement can be detected based on the change in color intensity of the pixels in the image determined by projecting onto a non-flat surface. It is also possible to first project a non-flat surface on the reference plane constructed in this way, and to perform reverse perspective mapping of the image by projecting onto this reference plane constructed according to optical rays. (Similar to normal flat ground).

  Reverse perspective mapping naturally corrects camera movement relative to the ground. In addition, it has the characteristic of not distorting objects and marks on the ground. However, the greatly distorted portion of the object on the ground appears to be represented in FIG. The image of the landscape (6) imaged by the camera (1) is as shown in FIG. 3A. A small object (7) on the ground, a mark (8) on the ground, a walking pedestrian (9), etc. Can be discriminated. FIG. 3B is a conversion image obtained by reverse perspective mapping. The object (7) and the mark (8) are not distorted, but the body of the pedestrian (9) appears to be greatly distorted.

  During the execution of step (a) of the method for detecting movement according to the present invention, the computing device of the on-board computer (5) has a corresponding series of consecutive landscape (6) images captured by the camera (1). Compute a series of images transformed by reverse perspective mapping based on. Based on the plurality of converted images, the arithmetic unit determines a background map by calculating a color intensity difference between each corresponding pixel in at least two converted images selected from the plurality of converted images, If the color intensity difference between corresponding pixels in the selected conversion image is sufficiently small, the background map is assigned by assigning an effective color intensity value that is close to the average color intensity of the corresponding pixels to the pixels in the background map. To decide. In this way, each pixel in the background map that is assigned an effective color intensity value constitutes a colored pixel. These are standard cumulative calculation steps for creating a background image, and a differential method for detecting a change in color intensity. However, such a process is performed on the converted image rather than on the image directly captured by the camera (1).

  The arithmetic device performs an area masking operation on the latest converted image among the obtained plurality of consecutive converted images, and stores the area masking data in the memory. To complete this step, the computing device of the on-board computer (5) takes into account received and stored stationary object data related to objects in the landscape that have been detected as stationary by the stationary object detection device. . This data represents a 2D shape (2D) or 3D shape (3D) that defines what is considered a boundary of a stationary object. The arithmetic unit uses this data to calculate which area of the converted image is masked by the detected stationary object. The arithmetic unit projects the received 3D shape data onto the ground, and uses the convex envelope around the projection of each shape as the 2D shape at the ground level. The area of the converted image located behind the 2D shape with respect to the position of these cameras (1) is masked. This masking can be said to build a blurry horizontal wall that stands in a 2D shape outline and use the projection of this wall in a reverse perspective mappin to mask out the area. If applicable, this masking can be limited to the detected object height (so as to limit the wall height) as long as the stationary object detector is a 3D detector. It is described in FIGS. 4A and 4B. FIG. 4A is an image taken by the camera (1), and is an image having a stationary object, that is, the vehicle (10). This vehicle is parked on the left side of the road. FIG. 4B is a conversion image obtained by reverse perspective mapping, and is obtained from the image of FIG. 4A. The trajectory (11) of the vehicle (10) and the area (12) of the conversion image hidden by the vehicle from the camera can be seen on the ground.

  Using the latest transformed image, the background map and the corresponding area masking data in the plurality of computed transformed images, the computing device may calculate the expected value being moved for each colored pixel in the background map. Establish a movement likelihood map associated with the pixel. To accomplish this, the computing device assigns a high likelihood of movement to each pixel where the corresponding colored pixel in the background map and the corresponding pixel in the final image have significantly different color intensities, and the computing device assigns a low likelihood of movement. Is assigned to each pixel where the color intensity of the corresponding colored pixel in the background map and the corresponding pixel in the final image are not very different. In addition, when there are pixels in the mask area, the arithmetic unit reduces the movement probability assigned to the pixels in this way. The movement prospect map obtained in this way is stored in the memory. In this image differentiation process, in order to avoid performing the calculation many times on the same pixel, for example, a flag is set on the pixels in the background map together with attributes (belonging to 3D obstacles or moving objects) to increase the processing speed. You can also.

  The move prediction operation can be flagged as one or more criteria, such as the history of each colored pixel (to detect whether the color density has changed over several images), the history of the pixel status (the nature of the obstacle). Or based on the current status of the pixel and also based on the previous status as the nature of the movement.

  The arithmetic unit performs filtering of the movement prospect map by determining, on the movement prospect map, a range associated with a sufficiently high movement prospect among the pixels. Then, the arithmetic unit stores the post-filtering movement likelihood map acquired in this way in the memory. Such determination of the relevant range can be performed using various well-known image segmentation techniques, some of which are described in detail in the following documents. Morel JM, Solimini S., “Variational Methods in Image Segmentation”, PNDLE, Vol. 14, Birkhauser, Basel (1994); Harlick RM, Shapiro LG, “Image segmentation techniques”, Computer Vision, Graphics and Image Processing, Vol. 29, p. 100-132, Jan. 1985. In the example described in this document, the arithmetic unit uses an aggregation type (blob) which is a well-known method described in detail in the following document. . R. Kauth, A. Pentland, G. Thomas, “Blob: an unsupervised clustering approach to spatial preprocessing of mss imagery” Proceedings of the 11th International Symposium on Remote Sensing of the Environment, Ann Harbor MI, p. 1309-1317, 1977 . A list of related ranges is created, and statistical calculations are performed on each range to reliably characterize their shapes. Fuzzy the calculated statistical results (to later use the fuzzy logic algorithm) and combine them together (with enough size) to calculate the likelihood of the relevant range belonging to a given type of obstacle For example, human movement can be detected by looking for a smaller relevant range and examining the relevant range extending in the direction of the camera or the direction of movement of the vehicle).

The arithmetic device determines whether or not the related range is related to the presence of the movement of the object based on a predetermined movement detection criterion in each related range of the calculated post-filtering movement expected map, and calculates a movement map. The calculated movement map is stored in the memory. The criteria for detecting movement in each relevant range are, for example, the surface area of the relevant range (to remove scattered image noise), the shape of the relevant range, the direction of the object of interest (here vertical in reverse perspective mapping) Objects are facing the camera.), Monitoring the relevant range in various cases (with Kalman filtering in the replacement of the relevant range), or the location of the relevant range (this criterion includes moving objects with specific parts of the map) This criterion is useful if not). Of course, some of these criteria may be integrated to generate a plurality of detection criteria. As one form of motion detection device related to the present invention, the arithmetic unit may calculate the reference plane of the previously acquired movement map. At the level, based on the calculation of the position of the optical center of the camera projected on the reference plane, each relevant range of the movement map will have the closest point of this projected optical center, i.e. the moving object on the reference plane. By determining each point determined corresponding to the contact point, an operation indicating the position of the moving object can be performed. The object position data calculated in this way is stored.

  As another form of the device for detecting movement according to the present invention, in the latest image of the plurality of converted images, the arithmetic unit may add a part of the reference surface based on the reference surface data stored in the memory. Corresponding pixels can be detected, and a zero movement likelihood can be assigned to each pixel of the movement likelihood map corresponding to the pixels detected corresponding to the reference plane. The movement map corrected in this way is stored in the memory. Thereby, the computing device of the on-board computer (5) can detect the ground in order to highlight the detected obstacle (stationary object or moving object) more clearly. Such detection of the ground can be performed using, for example, a known gray level range increasing method. In this method, the range increasing seed is selected in the transformed image, so if no movement is detected in the vicinity, the range increasing seed can be as close to the vehicle as possible. This method takes into account changes in landscape lighting conditions, changes in ground color, etc., using clay level thresholds that are updated during the increase (by adding pixels). This method is illustrated by the image of the repeated clay level range increase shown in FIG. A new pixel (initialized as a seed) that is a potential candidate for representing the ground is tested, and if it does not belong to a stationary or moving object, a transformed image that is added to the ground pixels detected during the iteration A binary map of the inner ground pixels is determined at the end of the process. As one form of the present invention, other known methods for detecting the ground are used. For example, a texture-based image segmentation method for determining similar texture areas (J. Zhang, H. Nagel, “Texture-based segmentation of road images”, Proceedings of IEEE Intelligent Vehicles Symposium). , p. 260-265, 1994). Or a method of detecting roads by grouping areas in the same direction (see Rasmussen C., “Texture-Based Vanishing Point Voting for Road Shape Estimation”, Proceedings of British Machine Vision Conference, 2004). .

  As another form of the apparatus for detecting movement related to the present invention, the arithmetic unit can detect a pixel corresponding to a part of the reference plane in the converted image before the latest converted image, and has high movement. Can be assigned to each pixel detected in the previous transformed image as having a non-zero movement probability in the movement likelihood map and belonging to the reference plane area. Further, the arithmetic unit stores the movement likelihood map thus corrected in the memory.

  FIG. 8 is an image showing various steps relating to movement detection in one form of the invention. FIG. 8 shows the current image (20) captured by the camera (1), for example, the last image of a plurality of consecutive images (21) captured by the camera. Image (22) shows the result of reverse perspective mapping of the current image (20). A ground detection process is performed on the image (23), and a stationary object masking process is performed on the image (24). The background map is obtained on the image (25), and the movement map is obtained on the image (26). As shown in the image (27), fuzzy logic filtering is performed on the movement map (26) to detect the relevant range corresponding to the pedestrian. Finally, image (28) is a moving map, and masked obstacles can be identified in image (29). The detected movement is identified in the image (30). The detected ground is identified by image (31) and the apparent movement after filtering is identified by image (32).

  Such a movement detection result can be displayed on a screen mounted on the vehicle. Based on the detection result, the driver (2) can determine the risk of collision and can drive the vehicle accordingly.

  The movement detection result by the movement detection device is transmitted to the collision prevention warning device. The arithmetic device can actually predict the risk of the vehicle (2) colliding with a stationary object or an object whose movement is detected, and sends the prediction result to the collision prevention warning device to warn of the collision. To do. The driver can determine the risk of collision with an object based on display data that is calculated and transmitted by, for example, the arithmetic unit of the collision prevention warning device and displayed on the screen. This collision risk display data has a colored area, which allows the driver to determine the risk of collision. For example, the color and color intensity of this area displayed on the screen allows the driver to intuitively determine the risk (for example, green indicates low risk and red or flashing indicates high risk). As shown).

  The image displayed on the screen may be transmitted by the arithmetic unit of the on-board computer (5). The colored collision risk display area is inlaid (PIP) into the image. Furthermore, the colored area that assists in predicting the risk of collision with the object may be expressed in a form corresponding to the locus of the object on the ground. According to this method, the driver can intuitively estimate the level of risk and can also perform an operation to avoid the object.

  The embodiment of the movement detection apparatus according to the present invention is the result of using the apparatus in various forms of the movement detection method according to the above-described invention, as in the other embodiments, and corresponds to the appended claims.

  According to the configuration of the present invention, it is not necessary to acquire knowledge of the type of object (shape or other properties) in advance, check the object, enormous calculation time or large calculation means, etc. It is possible to use a single landscape image acquisition device that can detect the movement of an object within the vehicle and that can move relative to the landscape.

It is the simplified explanatory drawing corresponding to the top view of the movement detection apparatus based on this invention provided with the camera mounted in the vehicle. 6 is a simplified aspect of reverse perspective mapping. It is the image converted by reverse perspective mapping. It is the image converted by reverse perspective mapping, and the figure corresponding to the stationary object is masked. It is an image showing the step of the detection process of the ground implemented in one Example of the detection apparatus of the object movement regarding invention. It is an image showing the step of the detection process of the ground implemented in one Example of the detection apparatus of the object movement regarding invention. It is an image showing the step of the detection process of the ground implemented in one Example of the detection apparatus of the object movement regarding invention. It is an image showing each step of the detection process of the movement implemented in one Example of the detection apparatus of the object movement regarding invention.

Explanation of symbols

1 Camera 6 Landscape 10 Still Object 12 Conversion Image Area 21 Image 26 Movement Map

Claims (10)

Based on the processing by the arithmetic unit of a plurality of scenery images taken by the camera (1) which is movable with respect to the reference plane, the detection method of the object movement in the scene (6) comprising a portion of the reference surface, the reference surface data for some shape and position of the reference plane is stored in the memory of the computing device, and the posture of the camera, set for each image of scenery that has been transmitted to the imaged the arithmetic unit by the camera calibration parameters are is stored in the memory of the computing device, the computing device can receive the stationary object data relating to an object that is stationary relative to a part of the reference plane detected in the landscape, the In the method for detecting object movement, wherein stationary object data is stored in the memory, and the stored data can be output .
Executed by the arithmetic unit,
For some of the reference plane, the reference plane data stored in said memory, corresponding pose of the camera, using the calibration parameters, captured by the camera, it is transmitted to the computing device, landscape Performing reverse perspective mapping to determine a plurality of corresponding transformed images in a plurality of consecutive images (21) of:
For each detected stationary object (10), on the basis and the stationary object data stored in memory corresponding pose parameters of the camera, converting the image corresponding masked with respect to the camera by the stationary object Determining an area (12), determining a background map (25) based on a change in color intensity between corresponding pixels in the converted image obtained in step (a), and masking the converted image area Taking into account this determined masked area (24) and a part of the background map, a movement map (26) that is constituted by pixels and determines whether it is related to the presence of object movement An object movement detection method comprising the step (a ′) of calculating . Wherein step (a ') further the likelihood value movement is taking place, the movement likelihood map by an association step (a) the pixels in said transformed image selected from the transformed image obtained in establishing, with respect to the pixel has a color intensity value corresponding pixels hardly vary image in the transmission image that was selected from among the obtained at step (a) image,
Moving expected to consider each area masked, it is associated with the pixel to the camera,
The higher the color intensity value of a corresponding pixel of said step (a) in the final in said transformed image obtained, differs from the color intensity value of a corresponding pixel in said selected transformed image,
2. Object movement detection according to claim 1 , wherein each pixel has a step corresponding to a pixel in the movement likelihood map that is determined to be associated with movement depending on the associated movement likelihood. Method. Wherein step (a ') is further performed by said arithmetic unit,
When the color intensity difference between corresponding pixels in the selected in the transformed image is sufficiently small at any time, the pixels in said background map corresponding to them, assigning a valid color intensity value close to the average color intensity value of the corresponding pixel it is said to determine the background map by calculating the color intensity difference between corresponding pixels of at least two said transformed image selected from a plurality transformed image obtained in step (a), the each pixel the effective color intensity value is assigned to the step (b) constituting the colored pixels,
Storing said obtained in step (a), of said transformed image in which a plurality of continuous, performs area masking operations on the end of said transformed image (22), the area masking data obtained in said memory (C),
Step at a plurality of said converted image obtained (a), the last of the transformed image, the background maps obtained in the step (b), corresponding area masking said obtained in step (c) data, on the basis of the movement likelihood map the mobile likelihood value corresponding is associated with each pixel for colored pixels in the background map,
Wherein the corresponding colored pixel in a background map, the pixel difference is large in color intensity between the corresponding pixels in the final image, assigning a high mobility likelihood,
Wherein the corresponding colored pixel in a background map, the pixel difference is small in color intensity between the corresponding pixels in the final image, assign the low mobility expected,
If if the pixel is located in an area masked, and storing this manner reduces the likelihood of a mobile assigned to pixels, the movement likelihood maps obtained in the memory (d),
In said mobile likelihood maps obtained in step (d), range associated within each pixel, by determining whether associated with sufficiently high mobility expected, the filtering processing on the moving prospective map performed, and step (e) for storing the filtering after the movement likelihood maps obtained in the memory,
To determine the moving map, for each relevant range of the after filtering movement estimated map is determined in the previous step, based on a predetermined moving direction reference, the relevant range is consistent with the presence of movement of the object The method for detecting movement of an object movement according to claim 1 or 2, further comprising the step (f) of determining whether or not. By the arithmetic unit, for each relevant range of the moving map the obtained in step (e), to determine the closest point of the projected optical center, the projection of the camera on the reference plane was based on the calculation of the position of the optical center C, it has a step (g) to determine the position of the moving object in the level of the reference surface on the moving map obtained in the step (f), thus detection method for object movement according to claim 3 each point is determined, characterized in that corresponding to the contact point of the moving object and the reference surface Te. Based on a plurality of successive said mobile maps obtained with the passage of the object movement detection methods time further determined at step (g), the successive positions of the corresponding representative points by using the operation device 5. The method of claim 4, comprising calculating (h) and monitoring (h) determining expected positions of these representative points, each of the representative points corresponding to a contact point of the moving object on the reference plane. Object movement detection method. Detection method for the object moving further following steps performed using the computing device, i.e. from the last image of the plurality of said converted image obtained in the step (a), in the memory based on the stored the reference surface data, a detection step of detecting a pixel corresponding to a portion of the reference surface,
Zero change expected, according to any one of claims 2 to 5 having an assignment step of the allocation to the mobile prospects each pixel in the map corresponding to the detected pixel as corresponding to the reference plane Detection method of object movement. Method for detecting the object movement is further executed by the computing device,
Last wherein in converting the transmitted image to the previous image of said detection step of detecting a pixel corresponding to a portion of the reference surface,
Claims, characterized in that it has an allocation step of allocating the high mobility expected to each pixel in said detected movement expected map as belonging to the reference plane in a previous said transformed image having a non-zero movement prospect 6. The object movement detection method according to 6. An apparatus for detecting the movement of an object in a landscape (6) having a part of a reference plane,
Movable relative to the reference plane, a mount comprising a position sensor which is capable of transmitting data defining the location of the basic spatial reference frame associated with the reference surface (4) (2),
Attached to said mount, landscape (6) is capable of capturing an image, and a predetermined posture parameters for the mount, the camera (1) capable of transmitting an imaging image with predetermined intraosseously calibration parameters,
The reference surface data for some shape and position of the reference surface, and mounting position data transmitted by the position sensor, the image data transmitted by the camera, the camera attitude parameters relating to the mount and the camera calibration parameters Data and a computing device (5) capable of receiving stationary object data relating to a stationary object detected in a landscape transmitted from an external environment,
The arithmetic device is capable calculating the orientation parameters of the Basic Camera reference frame, it is possible the processing of the received image data, a memory can store the received and calculated data can be transmitted stored data In the detection device,
The arithmetic unit, to obtain a plurality corresponding transformed image relates portion of the reference surface, the memory stored reference plane data, the attitude of the corresponding camera, by using a calibration parameter, from the camera Performing reverse perspective mapping of a plurality of consecutive landscape images (21) received (a) ,
For each detected stationary object (10), based on the corresponding camera posture parameter and the stationary object data stored in the memory, determines the corresponding transformed image area (12) to be masked relative to the camera by the stationary object And perform a masking process for this area (a ′) ,
Based on the calculation of the change in color intensity between the corresponding pixels of the transformed image that is stopped even step (a), considering each mask area determined, each pixel, there is movement of the object relative to the portion of the reference plane detecting device of an object movement calculates the moving map (26) configured by the associated pixel to, and having storing said moving map in the memory. The arithmetic apparatus further the likelihood value movement is taking place, said the association step (a) the pixels in said transformed image selected from the transformed image obtained in establishing the movement likelihood map, for that pixel has a color intensity value corresponding pixels hardly vary image in the transmission image that was selected from among the obtained at step (a) image,
The consideration of each area masked with respect to the camera, movement expected that are associated with the pixel, the higher the color intensity value of a corresponding pixel in the transformed image of the last of the obtained in step (a) is selected been different from the color intensity values of the corresponding pixels in said transformed image,
9. Object movement detection according to claim 8 , wherein each pixel has a step corresponding to a pixel in the movement likelihood map that is determined to be associated with movement depending on the associated movement likelihood. apparatus. The computing device further whenever the color intensity difference between the corresponding pixel in the selected the transformed image is sufficiently small, the pixels in the background map corresponding to them, close to the average color intensity value of the corresponding pixel by assigning a valid color intensity value, wherein by calculating the color intensity difference between corresponding pixels of at least two said transformed image selected from the plurality the transformed image obtained in step (a) (B) determining a background map, each pixel assigned its effective color intensity value constituting a colored pixel;
Storing said obtained in step (a), of said transformed image in which a plurality of continuous, performs area masking operations on the end of said transformed image (22), the area masking data obtained in said memory (C),
Step at a plurality of said converted image obtained (a), the last of the transformed image, the background maps obtained in the step (b), corresponding area masking said obtained in step (c) data, based on the movement likelihood map mobile likelihood value corresponding is associated with each pixel for colored pixels in the background map,
Wherein the corresponding colored pixel in a background map, the pixel difference is large in color intensity between the corresponding pixels in the final image, assigning a high mobility likelihood,
Wherein the corresponding colored pixel in a background map, the pixel difference is small in color intensity between the corresponding pixels in the final image, assign the low mobility expected,
If if the pixel is located in an area masked, and storing this manner reduces the likelihood of a mobile assigned to pixels, the movement likelihood maps obtained in the memory (d),
In said mobile likelihood maps obtained in step (d), range associated within each pixel, by determining whether associated with sufficiently high mobility expected, the filtering processing on the moving prospective map performed, and step (e) for storing the filtering after the movement likelihood maps obtained in the memory,
To determine the moving map, for each relevant range of the after filtering movement estimated map is determined in the previous step, based on a predetermined moving direction reference, the relevant range is consistent with the presence of movement of the object 9. The object movement detection device according to claim 8, wherein the step (f) of determining whether or not is performed.