JP2007089082A - Vehicular image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicular image display device provided with a constitution capable of quickly coping with a state when an image signal from an imaging camera is abnormal. <P>SOLUTION: The vehicular image display device 100 has an imaging means, which includes a plurality of the imaging cameras 102-108 for imaging the surrounding of a vehicle and outputs the image signals imaged by a plurality of the imaging cameras, an image processing means 110, which generates a composite image integrally showing the surrounding of the vehicle by using the image signals from the imaging means and outputs the image signal of the composite image, and a display means 116 for displaying the image signal of the composite image. The image processing means 110 displays a corrected image while quickly coping with image interference when it occurs by functioning as an image signal monitoring means for monitoring states of the image signals from the imaging cameras, and also, as an area changing means for changing an area adopting the image signals of the imaging cameras when confirming that any one of the image signals of the imaging cameras is abnormal. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image display device that combines the image signals from a plurality of imaging cameras mounted on a vehicle and displays the surroundings of the vehicle to improve driving safety and improve safety of the driver.

  With the improvement of the mounting rate of imaging cameras in automobiles, applications using image composition are rapidly expanding. Many of these applications assist the driver's driving and improve the safety of the vehicle during travel. For example, an in-vehicle image system that provides images of obstacles around a vehicle includes a camera system (such as a corner monitor system or a rear view camera system) that captures distant obstacles, and a system that captures blind spot obstacles in the vicinity of the vehicle ( Side view camera system). Further, in a system using an imaging camera with a narrow angle of view, the direction of the imaging camera can be changed by a driving device so that obstacles around the vehicle are projected. Furthermore, in a system using a plurality of ultra-wide-angle imaging cameras, the captured images are combined to project 360 degrees around the vehicle. This system cuts out or trims the required horizontal range of image data from the image captured by the imaging camera, and displays the composite image on the display.

  For example, Patent Literature 1 relates to a technique for obtaining a composite image that integrally displays the periphery of a vehicle using images from a plurality of imaging cameras, and proposes an image processing apparatus improved so as to obtain an appropriate composite image. When the imaging camera is installed near the door mirror mounting part or in a movable part such as a trunk, the composite image may be displaced when the door or trunk is opened, and the display image may give the driver a sense of incongruity. is there. Therefore, this image processing apparatus switches the display form of the composite image in accordance with the state of the movable part, and always displays the composite image without a sense of incongruity. Further, Patent Document 2 discloses an image composition device having the same function. This apparatus includes a changing unit that changes a range in charge of the imaging unit when an open / close body such as a door affects the imaging range of the imaging camera and a blind spot occurs in the imaging range.

JP 2001-334870 A JP 2004-48621 A

  Patent Document 1 and Patent Document 2 propose a technique for reducing an abnormality in a composite image that occurs when an opening / closing body such as a vehicle door or trunk is opened. Specifically, the present invention proposes a technique for improving so as not to display an image that gives the driver a sense of incongruity on the display. By the way, normally, when a door, a trunk, or the like is opened and closed, it is when the vehicle is stopped, and the driver recognizes that the door or the like is opened or closed. The opened door or the like is eventually closed and finally returns to a normal image. Therefore, even if the composite image is temporarily disturbed due to opening / closing of the door or the like, it does not become a serious problem when the driver operates the vehicle. On the other hand, when it is going to implement | achieve the technique currently disclosed by patent document 1 and patent document 2, various sensors which detect the state of the opening-closing body (moving parts, such as a door) of a vehicle, and the output of this sensor Therefore, a control system for performing image processing based on the above is required, so that the structure becomes complicated and the manufacturing cost increases.

  As described above, Patent Document 1 and Patent Document 2 are for preventing the display image from being disturbed in a short time when an opening / closing body such as a door or a trunk of a vehicle is opened, thereby eliminating a sense of incongruity felt by a driver or the like. It simply discloses the technology.

  By the way, as a case where the display image on the display is disturbed, it is assumed that some trouble occurs in the imaging camera. If a fault occurs in an imaging camera that is in charge of any area of the composite image, a shift occurs at the switching part (boundary part) between the images from the adjacent imaging camera, making it difficult to see the display, or there is a fault This may cause an image failure such that the area handled by the imaging camera cannot be confirmed at all. Thus, the image failure caused by the imaging camera itself is not temporary as in the case where the opening / closing body described above is opened, and seriously affects the driver (user). Therefore, prompt action is required. However, since the technique proposed in Patent Document 1 or Patent Document 2 does not consider the failure when the image signal from the imaging camera is abnormal, it cannot cope with this.

  Accordingly, the present invention has been made to solve the above-described conventional problems, and provides a vehicle image display device having a configuration for quickly dealing with an image signal from an imaging camera being abnormal. The main purpose.

  An image display device for a vehicle according to the present invention includes a plurality of imaging cameras that capture an image of the surroundings of the vehicle, uses an imaging unit that outputs image signals captured by the plurality of imaging cameras, and an image signal from the imaging unit. Image processing means for generating a composite image that integrally represents the periphery of the vehicle and outputting the image signal; and display means for displaying the image signal of the composite image; the image processing means further comprising the imaging camera Image signal monitoring means for monitoring the state of the image signal from the camera, and area changing means for changing the area where the image signal of the imaging camera is adopted when any of the image signals of the imaging camera is confirmed to be abnormal Is included.

  The image signal monitoring means compares the composite image immediately after the vehicle engine is started with the composite image immediately before the vehicle engine is stopped, and determines whether the image signal from the imaging camera is abnormal based on the difference between the two composite images. You may do it.

  The image signal monitoring means may determine whether or not there is an abnormality in the image signal from the imaging camera based on the occurrence of a shift in the boundary portion of the synthesized image. Note that an impact sensor or the like may be further provided in each imaging camera, and the image signal monitoring means may be activated based on the output of the sensor. In this case, it is possible to determine whether there is an abnormality in the image signal when an abnormality is likely to occur in the imaging camera.

  Further, the changing unit can correct the screen in which the abnormality has occurred by deleting or reducing the area where the abnormal image signal is employed and supplementing the area with the image signal from the adjacent imaging camera.

  Further, it is more preferable that the changing means performs a predetermined warning display on the display means when the image signal adoption area is changed.

  The image processing means may convert the composite image into an image signal as if looking down from a viewpoint above the vehicle.

  An image display method for a vehicle that generates and displays a composite image that integrally represents the periphery of a vehicle using image signals from a plurality of imaging cameras that capture the periphery of the vehicle, the image signal from the imaging camera A step of monitoring the state of the image capturing device, a step of changing an area where the image signal of the imaging camera is adopted when it is confirmed that any one of the image signals of the imaging camera is abnormal, and a step of displaying the changed composite image The above object can also be achieved by a vehicular image display method.

  In the step of monitoring the state of the image signal, the composite image immediately after the vehicle engine is started is compared with the composite image immediately before the vehicle engine is stopped, and the image signal from the imaging camera is compared based on the difference between the two composite images. You may make it determine the presence or absence of abnormality.

  According to the present invention, when the image signal from the imaging camera is abnormal, the area where the image is adopted is changed to switch to a display that reduces the influence of the image failure. Therefore, even if an abnormality occurs in the image from the imaging camera, the driver (user) can continue to support driving.

  The best mode for carrying out the present invention will be described below in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a vehicle image display apparatus according to an embodiment of the present invention. The vehicle image display apparatus 100 receives image signals of images captured by the imaging cameras 102, 104, 106, and 108 and the imaging cameras 102 to 108, and connects adjacent image signals to integrally surround the vehicle. The image processing unit 110 that generates a composite image represented by the following, and a display 116 that receives and displays an image signal of the composite image from the image processing unit 110.

  The imaging cameras 102 to 108 are configured using, for example, a CCD or a CMOS image sensor. For example, as shown in FIG. 2, the first imaging camera 102 is attached to a bumper or the like in front of the vehicle, and images the front of the vehicle with a viewing angle θ1. The second and third imaging cameras 104 and 106 are attached to both side pillars of the vehicle, and image the left and right sides with viewing angles θ2 and θ3. The second and third imaging cameras 104 and 106 may be installed on the left and right door mirrors. Further, the fourth imaging camera 108 is attached to a rear bumper or the like of the vehicle, and images the rear of the vehicle with a viewing angle θ4. The viewing angles θ1, θ2, θ3, and θ4 of the imaging camera are set so as to partially overlap each other with the adjacent viewing angles, so that the entire periphery of the vehicle can be imaged by the imaging cameras 102 to 108. Yes.

  Referring again to FIG. 1, the image processing unit 110 is configured using, for example, a microcomputer, and determines a method for generating a composite image that integrally represents the periphery of the vehicle using image signals from the imaging cameras 102 to 108. ROM, RAM, etc. for storing a program for converting to an image signal as seen from above, a program for dealing with any abnormality in the image signals from the imaging cameras 102 to 108, etc. Is included.

  The image processing unit 110 includes an image generation unit 112 and an output control unit 114. The image generation unit 112 receives image signals from the imaging cameras 102 to 108 and combines them to generate a combined image that integrally represents the surroundings of the vehicle, or looks down on this combined image from the viewpoint above the vehicle. Predetermined processing such as viewpoint conversion for display is performed. Further, the output control unit 114 checks the composite image signal generated by the image generation unit 112, and if there is an abnormality, controls the image generation unit 112 to correct the image signal and output it to the display 116 side. . The image generation unit 112 and the output control unit 114 will be further described.

  The image generation unit 112 generates a combined image that integrally represents the periphery of the vehicle by a predetermined standard combining method, and supplies the image signal to the output control unit 114. For example, as shown in FIG. 3, the standard combining method is to display an area in front of the vehicle with an image signal A imaged by the first imaging camera 102, and to display an area on the left and right sides of the vehicle. The image signals B and C captured by the second and third imaging cameras 104 and 106 are displayed, and the area behind the vehicle is displayed by the image signal D captured by the fourth imaging camera 108. As described above, the viewing angle θ of each of the imaging cameras 102 to 108 is set to be wide, and both ends of the images captured by the adjacent imaging cameras overlap. Accordingly, the image generation unit 112 generates a composite image signal that integrally represents the periphery of the vehicle by trimming the image signals of the overlapping portions. In addition, the image generation unit 112 receives image signals from the imaging cameras 102 to 108, converts the input image signals into digital signals, and also performs viewpoint conversion of the image signals for a look-down display. As shown in FIG. 3, the image signal A and the image signal B are combined at the boundary 120, the image signal A and the image signal C are combined at the boundary 122, and the image signal B and the image signal D are combined at the boundary. The image signal C and the image signal D are combined at the boundary 126. Note that the means for realizing image synthesis and viewpoint conversion processing by the image generation unit 112 may be either software or hardware.

  Since the image signals captured by the imaging cameras 102 to 108 include image signals of overlapping areas, the overlapping image signals are trimmed when the images are combined. In that case, if the image signal from one of the imaging cameras is abnormal, the display in the boundary region of the synthesized image becomes unnatural or discontinuous. In addition, all or part of the image disappears. If this is left unattended, the composite image displayed on the display 116 will be difficult for the user to see, and this will not be useful for improving driving support or safe driving.

  Therefore, the output control unit 114 monitors the state of the composite image generated by the image generation unit 112 and copes with this when an abnormality is confirmed. For example, the output control unit 114 compares the composite image immediately after starting the vehicle engine with the composite image immediately before stopping the vehicle engine, and determines whether there is an abnormality in the image signal from the imaging camera based on the difference between the two composite images. judge. If the composite image immediately after the engine is started is stored in a RAM or the like and compared with the composite image immediately before the engine is stopped, an abnormality of the imaging camera that has occurred while the vehicle is running can be confirmed. On the contrary, if the composite image immediately before the engine is stopped is stored in a RAM or the like and compared with the composite image immediately after the engine is started, an abnormality of the imaging camera that occurs after the vehicle is stopped can be confirmed. Of course, the output control unit 114 always stores the composite image immediately after the start of the engine and the composite image immediately before the engine stop, so that it is possible to check the abnormality of the imaging camera that has occurred while the vehicle is running or stopped.

  By the way, considering the case where the composite images are actually different, if the imaging camera collides with something and breaks, if it collides with something and the direction of imaging deviates, then the electrical connection such as disconnection of wiring etc. It is assumed that a failure occurs in the system, or that the lens is dirty and the expected imaging cannot be performed. When the output control unit 114 compares the composite image immediately after the engine start and the composite image immediately before the engine stop as described above and monitors the difference, the composite image is abnormal regardless of individual causes. Can be dealt with quickly.

  For example, when the deviation is confirmed at the boundary portion of the composite image, the output control unit 114 determines that the imaging camera in charge of the image area is out of order. Then, the composite image is changed so that the image handled by the imaging camera having the abnormality is reduced or deleted. The output control unit 114 supplies a signal for changing the composite image to the image generation unit 112, and the image generation unit 112 generates a composite image corrected based on the signal. The output control unit 114 supplies the image signal of the composite image thus changed to the display 116 for display.

  FIG. 4 is a diagram illustrating an example where the output control unit 114 changes the image adoption area. (A) shows a case where the output control unit 114 compares the composite images and determines that some abnormality has occurred in the image signal D from the rear imaging camera 108. At this time, the output control unit 114 reduces the area where the image signal D is adopted to a range where only the imaging camera 108 is imaged, for example, as shown in FIG. The area where the image signals B and C of 106 are employed is enlarged and supplemented. In this way, by changing the area where the image signal is adopted, the area where the image signal from the imaging camera 108 with the obstacle is reduced is reduced, so that the composite image displayed on the display 116 is difficult to see or inaccurate. Can be reduced.

  Further, for example, as shown in (c), the output control unit 114 can delete the image signal D from the imaging camera 108 and expand the area where the image signals B and C of the imaging cameras 104 and 106 on both sides are adopted. It supplements in the range. By not using the image signal from the imaging camera 108 with the obstacle, a part of the composite image is cut out, but it is preferable to displaying the inaccurate image as it is. And since a driver | operator is alerted, it contributes to safety improvement substantially.

  In the vehicular image display device according to the present embodiment, when the imaging camera fails, the output control unit 114 confirms this, and for example, as shown in FIGS. 4B and 4C, the composite image is displayed. Since changes are made and promptly dealt with, driving assistance to the driver can be continued. In addition, the change of the area shown in FIGS. 4B and 4C is an example, and is not limited to such a switching mode. If the defective part of the image from the imaging camera 108 can be grasped, it may be changed so as to remove the defective part. For example, when an image failure occurs around the left boundary portion 124 in the image signal D, the area in charge of the image signal D is not reduced symmetrically as shown in FIG. Alternatively, the image signal D of only the left part may be reduced, and the area in charge of the image signal B may be expanded to supplement this.

  FIG. 5 is a flowchart of an example executed when the output control unit 114 monitors an image signal from the imaging camera and finds an abnormality. FIG. 4 is a flowchart for monitoring an abnormality of the imaging camera that occurs while the vehicle is stopped. The output control unit 114 monitors whether or not the engine is stopped (step S101). When the engine stop is confirmed, the composite image generated by the image generation unit 112 using the image signals of the imaging cameras 102 to 108 at that time is stored in a storage device such as a RAM (step S102).

  Further, the output control unit 114 monitors whether or not the engine has been started next (step S103). When the engine start is confirmed, the output control unit 114 causes the image generation unit 112 to generate a composite image using the image signals from the imaging cameras 102 to 108 at that time, and the composite image when the engine is stopped stored in step 102. Are compared (step S104). The comparison here may be performed by comparing each of the image signals A to D to confirm whether or not there is a difference, or to confirm whether or not there is a shift in the boundary portion to which the image signal is connected. Etc.

  When the synthesized images are compared and there is a difference (step S105), the output control unit 114 changes the area in charge of the image signal from the imaging camera in charge of the image having the difference. Specifically, the output control unit 114 controls the image generation unit 112 to delete or reduce the image signal based on the imaging camera with the obstacle, and generate a composite image supplemented with the image signals from the imaging cameras on both sides. (See FIGS. 4B and 4C). The corrected image signal is supplied to the display 116 for display (step S106).

  According to the vehicle image display apparatus 100 described above, even when a failure or the like occurs in the imaging camera and a failure occurs in a part of the composite image, the display image can be corrected so as to minimize the influence. Therefore, even if an abnormality occurs in the image signal from the imaging camera, a composite image that integrally represents the surroundings of the vehicle useful for the driver can be continuously displayed.

  FIG. 5 shows a flowchart for monitoring a failure of the imaging camera that occurs when the vehicle is stopped. However, the failure of the imaging camera that occurs when the vehicle is running is changed as shown in FIG. Then, when the abnormality of the composite image is confirmed, it can be dealt with similarly. Furthermore, if the flowcharts shown in FIGS. 5 and 6 are combined, it is possible to monitor image failures that occur when the vehicle is traveling and when the vehicle is stopped, and to deal with it quickly.

  Next, a second embodiment of the present invention will be described. In the second embodiment, the vehicular image display device is applied to navigation. As shown in FIG. 7, the navigation system 200 includes a navigation function unit 210 that executes functions required for navigation, and an input unit 220 that receives input from the user, in addition to the vehicle image display device. The navigation function unit 210 calculates the host vehicle position using a GPS device or a self-contained navigation sensor, searches for an optimum route from the host vehicle position to the destination, and guides the searched route using the display 116. On the display 116, a road map around the vehicle position is displayed, and a mark indicating the vehicle position is synthesized and displayed there.

  The user can instruct the display mode on the display 116 via the input unit 220. For example, guidance by navigation can be displayed on the display 116 or can be switched to display a composite image around the vehicle. Alternatively, a composite image around the vehicle can be displayed on the display 116 as a two-screen display together with a navigation guidance display.

  In the above embodiment, an example in which a composite image that integrally shows the periphery of the vehicle is displayed using four imaging cameras is shown. However, the number of imaging cameras is not limited to this. For example, as shown in FIG. 8, eight imaging cameras are attached to the periphery of the vehicle, and each of the imaging cameras images a peripheral region 300, 302, 304, 306, 308, 310, 312, 314 of these vehicles, and these image signals May be synthesized. Furthermore, a composite image that is not displayed after the viewpoint conversion may be displayed on the display 116. Moreover, although the said Example illustrated about the case where the synthesized image which shows the whole periphery of a vehicle integrally is displayed, it is not restricted to such an aspect. For example, the image display apparatus for vehicles which displays the synthesized image which shows right-and-left both sides and back side integrally except the front side of a vehicle may be sufficient.

  In the above embodiment, when a failure of the imaging camera is confirmed and the assigned area is changed and the corrected composite image is displayed, a warning that the image from the imaging camera is abnormal is given to the user by voice or the like. Also good. Further, each imaging camera may be further provided with an impact sensor or the like, and the output control unit 114 may be activated based on the output of the sensor. In this way, it is possible to monitor whether an abnormality has occurred in the composite image when the imaging camera collides with something and receives an impact of a certain level or more. Therefore, it is possible to check the presence / absence of an abnormality in the composite image and respond quickly, particularly when the direction of the imaging camera is shifted or is highly likely to be damaged.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific embodiments according to the present invention, and various modifications can be made within the scope of the gist of the present invention described in the claims. Deformation / change is possible.

  The vehicle image display apparatus according to the present invention can be used as a safe driving support system or a parking support system while the vehicle is traveling. As a usage mode, it can be functionally coupled to an electronic device such as a navigation device or another computer.

FIG. 1 is a block diagram showing a configuration of a vehicle image display apparatus according to an embodiment of the present invention. It is a top view which shows arrangement | positioning of the imaging camera attached to the vehicle. It is a figure explaining the standard image composition method. It is the figure shown about an example in case an output control part changes an image employment area, (a) is that the output control part compares a synthesized image and some abnormality has generate | occur | produced in the image signal from the rear imaging camera. (B) is a diagram illustrating a case where an area where an image signal is employed is reduced, and (c) is a diagram illustrating a case where an area where an image signal is employed is deleted. It is the flowchart which showed the operation example of the image display apparatus for vehicles of a present Example. It is the flowchart which showed the other operation example of the image display apparatus for vehicles of a present Example. It is a block diagram which shows the structure of the navigation system which concerns on 2nd Example of this invention. It is a figure explaining the other example of attachment of an imaging camera.

Explanation of symbols

100: Image display device for vehicle 102, 104, 106, 108: Imaging camera 110: Image processing unit 112: Image generation unit 114: Output control unit 116: Display 118: Display 120, 122, 124, 126: Boundary part

Claims (9)

An imaging unit that includes a plurality of imaging cameras that image the periphery of the vehicle, and that outputs image signals captured by the plurality of imaging cameras;
Image processing means for generating a composite image integrally representing the periphery of the vehicle using the image signal from the imaging means, and outputting the image signal;
Display means for displaying an image signal of the composite image,
The image processing means further employs an image signal monitoring means for monitoring a state of an image signal from the imaging camera, and an image signal of the imaging camera when it is confirmed that any one of the image signals of the imaging camera is abnormal. An area changing means for changing the area to be
An image display device for a vehicle. The image signal monitoring means compares the composite image immediately after starting the vehicle engine with the composite image immediately before stopping the vehicle engine, and determines whether there is an abnormality in the image signal from the imaging camera based on the difference between the two composite images. Item 2. The vehicle image display device according to Item 1. The image display device for a vehicle according to claim 2, wherein the image signal monitoring unit determines whether there is an abnormality in the image signal from the imaging camera based on the occurrence of a shift in a boundary portion of the image to be synthesized. The vehicular image display device according to any one of claims 1 to 3, wherein the changing unit deletes or reduces an area where an abnormal image signal is used, and supplements the image signal with an adjacent imaging camera. The vehicular image display device according to claim 4, wherein when the image signal adoption area is changed, the changing unit performs a predetermined warning display on the display unit. The vehicle image display device according to claim 1, wherein the image processing unit converts the composite image into an image signal as if looking down from a viewpoint above the vehicle. A navigation system comprising: the vehicle image display device according to any one of claims 1 to 6; and navigation means capable of searching for a route to a destination and displaying the searched route on the display means. A vehicle image display method for generating and displaying a composite image that integrally represents the periphery of a vehicle using image signals from a plurality of imaging cameras that capture the periphery of the vehicle,
Monitoring the state of the image signal from the imaging camera;
Changing the area where the image signal of the imaging camera is adopted when confirming that there is an abnormality in any of the image signals of the imaging camera;
Displaying the modified composite image;
An image display method for a vehicle including: In the step of monitoring the state of the image signal, the composite image immediately after the vehicle engine is started and the composite image immediately before the engine stop of the vehicle are compared, and whether there is an abnormality in the image signal from the imaging camera based on the difference between the two composite images. The image display method for vehicles according to claim 8 which judges.

Images