JP4614852B2 - Vehicle surrounding monitoring method and apparatus - Google Patents

Description

  The present invention captures the periphery of the vehicle with a plurality of cameras mounted on the vehicle, and combines the captured images to display an image as if the surroundings of the vehicle were photographed from above the vehicle. Vehicle surroundings monitoring method that enables a driver to present information that can monitor the surroundings of a vehicle to a driver, such as being able to easily steer a vehicle into a narrow garage or parking lot, and the like It relates to an apparatus for carrying out the method.

  When putting a vehicle in a narrow garage or parking lot, the driver needs to be very careful and sometimes accidents such as rubbing the surrounding vehicle with the surrounding object may occur. Also, when taking a vehicle out of a garage or parking lot, it is necessary to pay close attention to surrounding objects and people. In addition to entering and exiting garages and parking lots, vehicles such as driving on extremely narrow roads or contacting with surrounding objects when passing on opposite roads on narrow roads, or preventing wheels from falling into grooves, etc. There are many cases where sufficient attention is required around the area.

  As countermeasures, the rear camera that has been widely installed in vehicles in recent years is photographed and displayed on the monitor to facilitate garage and parking, and both front sides of the vehicle are photographed. When the vehicle exits a narrow road with poor visibility, use a camera for taking a side view of the vehicle to check the vehicle traveling on the other road, etc. It is also done while driving.

  However, even if such an outside-camera camera is used as an ultra-wide-angle camera, for example, when a garage is placed while the vehicle is retracted, and there is an obstacle around the rear of the vehicle, the rear-facing camera will Being out of sight and unable to know the state of the obstacle. As a countermeasure, it is also conceivable to provide super wide-angle cameras on both sides of the vehicle so that the portion outside the field of view of the rear photographing camera can be seen by the side photographing camera.

  However, in the image of the ultra-wide-angle camera as described above, the appearance of the object differs greatly between the central portion and the peripheral portion of the captured image, and simply looking at the captured image shows the positional relationship between the object and the vehicle. It is not clear and may cause unexpected collisions and rubbing. In particular, even if the image from the rear camera is switched to the image from the side camera, the image of the object such as an obstacle displayed on the screen at that time is very different, and the actual state of the object must be grasped. Is not easy.

  Therefore, a plurality of cameras for shooting outside the vehicle are mounted on the vehicle, the images shot by these cameras are combined, and an image is formed as if the surroundings of the vehicle were shot from the camera installed above the vehicle. It has been proposed to be displayed. When using such a plurality of cameras, a rear camera for photographing the rear of the vehicle, a right camera for photographing the right side of the vehicle, a left camera for photographing the left side of the vehicle, the front side of the vehicle It is equipped with a total of four cameras for the front of the vehicle that shoot the camera, combines all the images from these cameras, and seems to shoot the entire periphery of the vehicle with the camera installed directly above the vehicle It is also proposed to display on a monitor.

Note that a technique for displaying a marker on a screen for a target with a high ground in an image of a rear-view camera is disclosed in Patent Document 1 below, and a vehicle is photographed and synthesized by a plurality of cameras. A technique for setting the shooting direction of a camera so that an object in the shooting boundary area of a plurality of cameras is not visible when an image is obtained is disclosed in Patent Document 2 below.
JP 2004-56219 A JP 2003-204547 A

  As described above, when a plurality of cameras that capture the periphery of the vehicle are used and the captured images are combined, an image that is captured by the camera around the vehicle from above the vehicle is displayed. Although captured images of surrounding objects are displayed without a sense of incongruity, the vehicle surroundings monitoring device can be made easy to see, but the following problems arise.

  That is, for example, as shown in FIG. 5, the rear camera B after the vehicle 1, the front camera F in front, the right camera R for photographing the right side portion of the vehicle, and the left portion for photographing the left side portion of the vehicle. When the four cameras of the camera L are provided to shoot the surroundings of the vehicle, and the images taken by these cameras are combined to display an image of shooting the surroundings of the vehicle by the camera provided above the vehicle. When the vehicle moves backward by slowing down to enter the garage, the state when the plane mark M exists diagonally rearward on the right side of the vehicle is shown in (a) to (c), and the ball is in the same position. (B) (a) to (c) show the state when the spherical object O is present, and the state of the composite image in the state where the pole-like object P higher than the spherical object exists at the same position ( c) (a) to (c).

  As shown in FIG. 5 (a), when a plane mark M is present obliquely rearward on the right side of the vehicle, it is first photographed by the rear camera B as shown in FIG. 5 (a), and the photographed image at that time is shown in FIG. As shown in the figure, this mark is displayed in a circle as when viewed with a camera provided above the vehicle. After that, as shown in (b), the vehicle moves backward, and after reaching the limit position where the rear camera B can shoot, it enters the shooting range of the right side camera as shown in (c). Therefore, the mark M is continuously displayed on the monitor screen. Therefore, since the composite image of the mark M on the plane has little shape change after the viewpoint conversion due to the movement of the vehicle, the mark M having the same shape is always displayed continuously.

  On the other hand, as shown in FIG. 5B, when a spherical object O such as a ball exists at the same position as described above, the image captured by the rear camera B and the image captured by the right camera R are combined. The obtained image is formed by the rear camera photographed image range BG by the rear camera B and the right camera photographed image range RG by the right camera R.

  The captured image of the three-dimensional object having the height as described above causes a shape change after the viewpoint conversion as the vehicle moves. Therefore, in particular, as shown in FIGS. 5B and 5C, the composite image boundary line BRGL of the rear camera photographed image range BG and the right camera photographed image range RG is displayed in a small circular shape. Become.

  Further, as shown in FIG. 5C, when there is a pole-like object P having a relatively high height at the same position as described above, in order to form an image viewed from above as this image composition method, By converting under conditions that match the height, all the height information is added on the plane, resulting in a phenomenon that the area becomes wider.

  For this reason, the view of the three-dimensional object becomes unnatural, or, for example, as shown in FIGS. 5C and 5C, the composite image boundary line BRGL portion of the rear camera captured image range BG and the right camera captured image range RG Then, almost all objects disappear. For this reason, it is inevitable that the driver who operates the steering wheel by trusting the display on the monitor screen will be in a dangerous state.

  Therefore, according to the present invention, the vehicle is photographed by photographing the surroundings of the vehicle with a plurality of cameras and combining the photographed images so that the surroundings of the vehicle are photographed by the camera provided above the vehicle. In a vehicle surroundings monitoring device that presents the state of surrounding objects to the driver, etc., the three-dimensional object is almost lost at the composite image boundary part of the images captured by the multiple cameras, and the presence of the three-dimensional object cannot be determined. The main purpose is to prevent troubles.

In order to solve the above-described problem, a vehicle periphery monitoring method according to the present invention inputs a plurality of camera images for photographing the outside of the vehicle, creates a composite image with the upper side of the vehicle as a viewpoint, and exceeds a predetermined value on the composite image. An object image of the size of the vehicle, detecting the movement of the vehicle, calculating the movement length in the composite image corresponding to the movement of the vehicle, Then, an object having a portion moved more than the movement length in the calculated composite image is detected as a three-dimensional object, and a marker is displayed on the image portion with the least movement in the detected three-dimensional image, and the marker is displayed. When the three-dimensional object image is present in the composite image, it is displayed on the three-dimensional object base, and when the three-dimensional object base image disappears, the image is moved and displayed in accordance with the movement of the vehicle. To do.

In order to solve the above problems, the vehicle periphery monitoring device according to the present invention inputs the images of a plurality of cameras that photograph the outside of the vehicle, and creates a vehicle upper viewpoint image composition means for creating a composite image with the viewpoint above the vehicle. Corresponding to the movement of the vehicle detected by the vehicle movement detection means, a predetermined object detection means for detecting an object image of a predetermined size or more on the composite image, a vehicle movement detection means for detecting the movement of the vehicle, In-image movement length correspondence calculation means for calculating the movement length in the composite image, and in-image movement length correspondence calculation of the vehicle movement among the object images detected by the predetermined object detection means A three-dimensional object detecting means for detecting an object having a portion moved more than the image length calculated by the means as a three-dimensional object, and an image portion having the least movement in the three-dimensional object image detected by the three-dimensional object detecting means. A three-dimensional object base marker display means for displaying a marker, wherein the three-dimensional object base marker display means displays a marker on the three-dimensional object base when an image of the three-dimensional object exists in the composite image, and the three-dimensional object base When the image disappears, the marker is moved and displayed in accordance with the movement of the vehicle.

  In another vehicle surrounding monitoring apparatus according to the present invention, the vehicle movement detecting means detects the movement of the vehicle based on the moving distance of the vehicle, and calculates the movement length in the image corresponding to the movement of the vehicle. The means calculates a moving length in the composite image corresponding to the moving distance of the vehicle.

  Further, in another vehicle surrounding monitoring apparatus according to the present invention, in the vehicle surrounding monitoring apparatus, the vehicle movement detecting means detects movement of the vehicle based on a moving distance of the vehicle and a steering wheel rotation angle of the vehicle, and The in-image moving length correspondence calculating means calculates the moving length of each part in the composite image based on the moving distance of the vehicle and the steering wheel rotation angle of the vehicle.

  Another vehicle surrounding monitoring apparatus according to the present invention is characterized in that, in the vehicle surrounding monitoring apparatus, the camera that images the outside of the vehicle is a camera that also detects infrared rays.

  Since the present invention is configured as described above, the camera is photographed as if the surroundings of the vehicle were photographed by the camera provided above the vehicle by photographing the surroundings of the vehicle with a plurality of cameras and combining the photographed images. In the vehicle surroundings monitoring device that presents the state of objects around the vehicle to the driver and the like by displaying the three-dimensional object even when the three-dimensional object almost disappears at the composite image boundary portion of the images captured by the plurality of cameras. Since the marker is displayed at the position where the base of the object should be present, the driver can prevent the danger of collision with the three-dimensional object because the existence of the three-dimensional object is not known.

  In the present invention, when a vehicle is photographed with a plurality of cameras and an image with the viewpoint above the vehicle is synthesized and displayed, the three-dimensional object is almost disappeared at the boundary portion of the synthesized image of the images photographed by the plurality of cameras and cannot be seen. The problem is that a plurality of camera images that capture the exterior of the vehicle are input to create a composite image with the top of the vehicle as a viewpoint, and an object image of a predetermined size or larger is detected on the composite image, The movement is detected, the movement length in the composite image corresponding to the movement of the vehicle is calculated, and the calculated movement length in the composite image among the detected object images of a predetermined size or larger is calculated. The object with the moved part is detected as a three-dimensional object, a marker is displayed on the image part with the least movement in the image of the detected three-dimensional object, and the marker is displayed on the synthesized image. Regardless, it was solved by moving the display in response to movement of the vehicle.

  Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a functional block diagram of an embodiment of the present invention. In addition, each function part in the same figure can also be called means to perform each function. In the embodiment shown in FIG. 1, in order to photograph the surroundings of the vehicle 1, a left-side camera 2 that is disposed on the left side with respect to the traveling direction of the vehicle and mainly photographs the lower left side, and a right-side camera that is disposed on the right side. Four cameras are mounted: a right side camera 3 that captures the lower part, a rear camera 4 that is disposed at the rear of the vehicle and mainly captures the rear, and a front camera 5 that is disposed at the front of the vehicle and primarily captures the front. An example is shown. As these cameras, cameras capable of photographing a sufficiently wide angle are used.

  In the vehicle surrounding monitoring apparatus shown in FIG. 1, the left camera image input unit 7, the right camera image input unit 8, the rear camera image input unit 9, and the front camera image input unit 10 in the outside camera image input unit 6. The image of each camera is input in. The images of these cameras are synthesized in the vehicle upper viewpoint image synthesis unit 11 as if the camera is installed above the vehicle and the surroundings of the vehicle are photographed. In such image synthesis, various methods such as a method for which a development result has already been reported can be employed.

  FIG. 3 shows an example in which the shooting ranges of the four cameras provided in the vehicle 1 shown in FIG. 1 and images shot by these cameras are combined. In the example of FIG. Shows a left side mirror provided on the left side of the vehicle, the right camera 3 is similarly installed on the right side mirror, the rear camera 4 is installed on the rear of the vehicle body, and the front camera 5 is installed on the front bumper. However, these are only examples, and each camera can be mounted at any position according to the shape of the vehicle body, etc., so that the shooting range of these cameras can be taken overlapping each other, especially in the proximity of the vehicle It is installed so that you can know the surrounding conditions.

  FIG. 3C shows an example of the result of combining the images taken from the above-described images with the images viewed from above the vehicle, such as when the camera is installed above the vehicle and the periphery of the vehicle is photographed. This image is obtained by the same image processing as the conventional example shown in FIG. That is, the rear camera captured image range BG based on the captured image of the rear camera 4, the front camera captured image range FG based on the front camera 5, the left side camera captured image range LG based on the left side camera 2, and the right side camera captured image based on the right side camera 3. A predetermined image around the vehicle 1 is obtained by the range RG. Such image composition is performed in the vehicle upper viewpoint image composition unit 11 in FIG. 1, thereby obtaining the vehicle upper viewpoint image GG.

  The mark addition to the stereoscopic image and the movement process of the mark according to the present invention are performed in the vehicle upper viewpoint composite image GG, and the state has been described in correspondence with the prior art of FIG. Is FIG. (A) to (c) in FIG. 4A are a rear camera B after the vehicle 1, a front camera F in front, a right side camera R for photographing a right side portion of the vehicle, and a left side side of the vehicle. The left camera L that shoots the part is provided with four cameras to photograph the surroundings of the vehicle, and the captured images of these cameras are combined to photograph the surroundings of the vehicle with the camera provided above the vehicle. This indicates whether to display a correct image. In this state, when the vehicle moves backward by slowing down because it enters the garage, the state when the flat mark M exists diagonally rearward on the right side of the vehicle is shown in (a) to (c). As shown in (a) to (c) of (b) and (c), a pole-shaped three-dimensional object is present at the position of. In addition, since (a) of FIG. 4 was already demonstrated, the duplication description here is abbreviate | omitted.

  In the predetermined object detection unit 12 of FIG. 1, an object image having a predetermined length or more is detected by detecting an object image having a predetermined length or more from the images combined by the vehicle upper viewpoint image combining unit 11. A possible object image is detected. That is, as described in the explanation of FIG. 5 (c), when there is a relatively high object, the height is adjusted to the ground to form an image viewed from above as this image composition method. Therefore, all the height information is added on the plane and expressed as the length on the plane.

  Therefore, by detecting an object having a predetermined length on the screen, it is possible to recognize an object that is high enough to be an obstacle for the vehicle. However, at the first stage when such an image appears, it is not known here whether the image is an image like a simple mark on a plane or an image of a three-dimensional object as described above.

  In FIG. 1, a vehicle travel detection unit 13 inputs a travel distance such as a travel distance such as a travel distance within a predetermined time and a vehicle travel distance input unit 14 for inputting a travel distance such as a travel distance within a predetermined time. A vehicle handle rotation angle input unit 15 that detects the angle of the rotation operation is provided, and the signal is output to the in-image movement length correspondence calculation unit 16 of vehicle movement. In the in-image movement length correspondence calculation unit 16 of this vehicle movement, most basically, when the vehicle has moved, for example, 30 cm, the predetermined distance indicating how long the movement distance corresponds in the composite image is determined. It is calculated by multiplying the coefficient.

  Furthermore, more precisely, when the driver operates the steering wheel to adjust the traveling direction of the vehicle at that time, the direction of shooting with the camera changes accordingly, which is different from when the vehicle is simply moving in a straight line. Since the image of the object displayed on the screen rotates and moves, correction calculation is performed using the signal from the vehicle handle rotation angle input unit 15 in order to consider the movement state.

  The calculation result is output to the three-dimensional object detection unit 17, and for an image that may be a three-dimensional object that becomes a detection obstacle in the predetermined object detection unit 12, the length in the image corresponding to the vehicle travel distance is L1. At this time, it is determined by calculating whether or not there is a part that has moved by a length L1 or more corresponding to the travel distance of the vehicle in an image that may be a three-dimensional object. For example, as shown in (b) of FIG. 4B, when there is an object image having a length L2 longer than the length L1, the image is not a planar image and is likely to become an obstacle. It can be seen that the image is a three-dimensional object having a size.

  When the three-dimensional object image is detected by the three-dimensional object detection unit 17 as described above, the three-dimensional object base detection unit 18 sets a movement length L1 portion corresponding to the moving distance of the vehicle in the three-dimensional object image on the ground of the three-dimensional object. It is determined that the portion is closest to the vehicle, and that portion is detected as the base of the three-dimensional object. In the three-dimensional object base marker display unit 20, for example, an L-shaped or inverted L-shaped mark A with a color that is easy to see is given to the three-dimensional object base detected by the three-dimensional object base detection unit 18. This state is shown in (b) of FIG.

  The marker movement calculation unit 19 corresponds to the subsequent movement of the vehicle input by the vehicle travel distance input unit 14, and further corresponds to the steering wheel rotation angle of the vehicle as necessary, and the movement length in the image of the vehicle movement. Based on the result calculated by the correspondence calculation unit 16, the marker displayed as described above is moved within the screen. As a result of such processing, the images are displayed as shown in FIGS. 4C and 4C, and the conventional example shown in FIG. 5, that is, as shown in FIGS. In the present invention, even if the three-dimensional object image disappears at the connection portion from the photographed image range to the photographed image range of another camera, the marker A does not disappear in the present invention, and it corresponds in the screen as the vehicle moves. The driver can safely move backward while paying attention to obstacles.

  The vehicle surrounding monitoring apparatus according to the present invention as shown in FIG. 1 can be implemented by, for example, sequentially operating according to the operation flow shown in FIG. In the display processing of the three-dimensional object marker by the outside-the-car shooting composite image shown in FIG. 2, first, four sides are shot with a plurality of the outside-camera shooting cameras (step S1). Next, a vehicle upper viewpoint image in which the vehicle is viewed from above is synthesized by images taken by the cameras (step S2). This processing is performed in the vehicle upper viewpoint image composition unit 11 in FIG.

  Thereafter, it is determined whether or not there is an object image having a size larger than a predetermined size in the captured image (step S3). Here, an image of a predetermined size or more is an image that is taken with a camera and combined, and for example, if a three-dimensional object with a height of 30 cm or more is dangerous, how much such a three-dimensional object is in a planar image. It can be determined by converting whether the length has been expanded. However, here, even if an object image of such a size exists, it is unknown whether it is a planar object or mark or a three-dimensional object as described above. Yes.

  As a result of the determination in step S3, when it is determined that there is no object image larger than a predetermined size in the image, it is determined whether or not the vehicle is currently moving (step S14), and it is determined that the vehicle is moving. Sometimes the process returns to step S3 again to make the same determination. When it is determined in step S14 that the vehicle is not currently moving, it is determined whether or not the engine is stopped (step S15). If it is determined that the engine has been stopped, the process proceeds to step S16 and the process is terminated. If it is determined that the engine has not yet stopped, the process returns to step S14 to determine whether or not the vehicle is moving and the operation is repeated.

  As a result of the determination in step S3, when it is determined that an object image having a size larger than a predetermined size is present in the image, if the vehicle subsequently moves (step S4), the travel distance of the vehicle is measured (step S5), and further the vehicle The handle rotation angle is measured (step S6). Next, it is determined whether or not there is an object image having a portion that has moved beyond the length corresponding to the vehicle movement in the composite image (step S7). In this process, the three-dimensional object detection unit 17 in FIG. 1 compares the calculation data by the in-image movement length correspondence calculation unit 16 of the vehicle movement and the movement length of a part of the object image detected by the predetermined object detection unit 12. Is done. In that case, it is preferable to calculate the movement length in consideration of the fact that the captured image of the camera is rotated by the change in the direction of the vehicle due to the rotation of the vehicle handle, in addition to the travel distance of the vehicle as described above.

  Next, when it is determined in step S7 that such an object image does not exist, it is determined that an object of a predetermined size or larger in the captured image determined in step S3 is not a three-dimensional object, and the process returns to step S3. It is determined again whether or not there is an object image of a predetermined size or larger in the captured image, and the same operation is repeated thereafter. When it is determined in step S7 that such an object image exists, it is detected that the object of the image is a three-dimensional object having a size greater than or equal to a predetermined size, and a marker is located at a portion equal to the vehicle movement distance in the detected three-dimensional object. Display.

  That is, when it is determined in step S7 that the object image is an image of a three-dimensional object, a portion equal to the moving distance of the vehicle in the image of the three-dimensional object is assumed to be a base part in contact with the ground. The three-dimensional object base marker display unit 20 in FIG. 1 displays the marker A as shown in (b) of FIG. Various shapes of the marker A can be used as long as they are easy for the user to see. When there are a plurality of three-dimensional objects as described above, a marker is displayed on each of the bases. In that case, it can implement in various aspects, such as changing the color of each marker.

  Thereafter, it is determined whether or not the vehicle is currently moving (step S9). When it is determined that the vehicle is not moving, it is determined whether or not the engine is stopped as in step S15 (step S13). If it is determined that the engine has been stopped, the operation flow ends (step S16). If it is determined that the engine has not been stopped, the process returns to step S9 and the above operation is repeated.

  When it is determined in step S9 that the vehicle is moving, in the illustrated embodiment, it is determined whether or not the recognized solid object displaying the marker is displayed on the screen (step S10). As a result of the determination, when it is determined that the recognized three-dimensional object displaying the marker is displayed, the marker is moved and displayed with respect to the base of the recognized three-dimensional object displaying the marker (step S12).

  As a result of the determination in step S10, when it is determined that the recognized three-dimensional object displaying the marker is not displayed, the marker is moved and displayed on the screen by an amount corresponding to the moving distance of the vehicle (step S11). In other words, even if a phenomenon occurs in which the image of a three-dimensional object disappears at the joint between the captured image range of one camera and the captured image range of another camera connected thereto, only the marker is the vehicle. Since the vehicle moves in response to the movement, the driver may move the vehicle while paying attention to the movement of the marker. After the marker is moved in step S11, the process returns to step S9 again to determine whether or not the vehicle is currently moving, and the same operation as described above is repeated. Thereafter, if it is determined in step S12 that the vehicle is stopped as described above, and it is determined in step S13 that the engine is stopped, the operation flow is ended in step S16.

  The monitoring of the surroundings of the vehicle as described above operates in the same manner when the vehicle moves forward as well as when the vehicle moves backward as in the above embodiment, and an infrared camera is used to monitor the three-dimensional object around the vehicle at night. It is also possible to combine the images in the same manner as described above and perform the same operation. In that case, the camera shooting mode can be switched to the infrared camera mode as in a general home video camera, and in that case, as in the home video camera, the surroundings are irradiated with infrared rays and shot. It is preferable.

  The present invention can be used in various vehicles including special vehicles in addition to being used as a surrounding monitoring device for general vehicles.

It is a functional block diagram of the Example of this invention. It is an operation | movement flowchart of the Example. The figure which shows the example which mounts the camera for vehicle outside photography in the Example, image | photographs the circumference | surroundings, and synthesize | combines the image which image | photographs the circumference | surroundings of the vehicle with the camera provided in the upper part of the vehicle by the picked-up image. is there. It is explanatory drawing of the Example which provides a marker to the base of the image of a solid object in this invention. It is a figure which shows the example in which a ground marker, a spherical object, and a pole-like object are displayed by the synthesized image in the conventional apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle 6 Camera outside camera image input part 11 Vehicle upper viewpoint image synthetic | combination part 12 Predetermined object detection part 13 Vehicle travel distance input part 14 Vehicle travel distance input part 15 Vehicle handle rotation angle input part 16 Intra-image movement length of vehicle movement Corresponding calculation unit 17 Solid object detection unit 18 Solid object base detection unit 19 Marker movement calculation unit 20 Solid object base marker display unit

Claims (5)

Input images from multiple cameras that capture the exterior of the vehicle and create a composite image with the viewpoint above the vehicle.
Detecting an object image of a predetermined size or more on the composite image;
A movement of the vehicle is detected, a movement length in the composite image corresponding to the movement of the vehicle is calculated, and a movement in the calculated composite image is detected among the detected object images of a predetermined size or larger. Detect an object with a part that has moved more than the length as a three-dimensional object,
In the detected three-dimensional object image, a marker is displayed on the image portion with the least movement,
The marker is displayed on the three-dimensional object base when the image of the three-dimensional object is present in the composite image, and is moved and displayed corresponding to the movement of the vehicle when the image of the three-dimensional object base disappears. A vehicle surrounding monitoring method characterized by the above. Vehicle upper viewpoint image synthesizing means for inputting a plurality of camera images for photographing the outside of the vehicle and creating a composite image with the upper side of the vehicle as a viewpoint;
A predetermined object detecting means for detecting an object image of a predetermined size or more on the composite image;
Vehicle movement detection means for detecting movement of the vehicle;
In-image movement length correspondence calculation means for calculating the movement length in the composite image corresponding to the movement of the vehicle detected by the vehicle movement detection means;
Three-dimensional object detection for detecting, as a three-dimensional object, an object having a portion that has moved more than the in-image length calculated by the in-image moving length correspondence calculating means of the vehicle movement in the object image detected by the predetermined object detecting means. Means,
A three-dimensional object base marker display means for displaying a marker on an image portion having the least movement in the three-dimensional object image detected by the three-dimensional object detection means;
The three-dimensional object base marker display means displays a marker on the three-dimensional object base when the image of the three-dimensional object exists in the composite image, and responds to movement of the vehicle when the image of the three-dimensional object base disappears. A vehicle surroundings monitoring device characterized by moving and displaying a marker. The vehicle movement detection means detects the movement of the vehicle based on the movement distance of the vehicle, and the movement length correspondence calculation means in the image of the vehicle movement calculates a movement length in the composite image corresponding to the movement distance of the vehicle. The vehicle surrounding monitoring apparatus according to claim 2, wherein The vehicle movement detection means detects the movement of the vehicle based on the movement distance of the vehicle and the steering wheel rotation angle of the vehicle, and the calculation means corresponding to the movement length in the image of the vehicle movement determines the movement based on the movement distance of the vehicle and the steering wheel rotation angle of the vehicle. The vehicle surrounding monitoring apparatus according to claim 2, wherein a moving length of each part in the composite image is calculated. 3. The vehicle surrounding monitoring apparatus according to claim 2, wherein the camera for photographing the outside of the vehicle is a camera that also detects infrared rays.

Images