JP2012105090A - Parking support apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a parking support apparatus capable of simply correcting difference in color between photographic images from plural cameras, without needing a color reference member etc. to be mounted to a vehicle.SOLUTION: The parking support apparatus extracts a specific road sign such as a white-based road sign from images photographed by the cameras which photograph the surrounding of the vehicle with respectively different photographic visual fields, and performs image correction such as white balance on the photographed images on the basis of the specific road sign extracted from the photographic images, thereby generating corrected photographic images. The parking support apparatus composes bird's-eye-view image segments of each camera generated from the generated corrected photographic image through visual-point conversion, thereby generating a bird's-eye-view image as a display image of the surrounding of the vehicle to be displayed on a monitor.

Description

  The present invention relates to a parking assistance device that supports parking by displaying, on a monitor, overhead images generated based on images taken by a plurality of cameras that photograph the periphery of a vehicle with different field of view.

  An overhead image segment that is an image viewed from above the vehicle is generated from the images taken from the four cameras, and the four overhead image segments are combined with a mark indicating the vehicle stored in advance and output to the monitor. An apparatus is known from US Pat. In this apparatus, a conversion formula for correcting a distortion or brightness difference of a captured image acquired by the camera is set in advance for each position in the captured image based on the characteristics of the lens system. Thus, the captured image is corrected, and the same object and the same texture are always displayed in the same manner in the captured image regardless of the position in the imaging range of the camera. However, since this apparatus uses a conversion formula set in advance for each camera, brightness correction for each camera can be performed satisfactorily, but the color balance in the captured image between the cameras caused by the environmental light source can be adjusted. I can't. Therefore, the rear camera shoots the surrounding scenery with the brake lamp as the main light source, the front camera shoots the surrounding scenery with the front lamp as the main light source, and the side camera shoots the surrounding scenery with the parking lot lamp as the main light source. If a bird's-eye view image is created based on the images taken from the respective cameras, the color balance in the bird's-eye view image is greatly lost, and the image is difficult to see for the driver.

  In general, a technique called white balance adjustment is known to solve the problem of color cast caused by a light source. This white balance adjustment is one of image processing techniques for adjusting a captured image so that a display color recognized when a human directly looks at an object to be captured is also accurately reproduced in the captured image. For example, as an in-vehicle image recognition device employing such a white balance adjustment technology, a dominant light source outside the vehicle (based on a color temperature estimated from a photographed image of an achromatic member disposed at a predetermined position in the vehicle) A technique for estimating the color temperature of the main light source and reducing the influence of the light source on the captured image is known from Patent Document 2. In other words, in order to estimate the color temperature of the light source outside the vehicle based on the captured image captured by the imaging unit mounted on the vehicle, an achromatic member is provided at a predetermined position in the vehicle. An image of the periphery of the vehicle is taken so as to include the sky. Estimate the color temperature of the image area of the achromatic member in the captured image, and determine the color temperature of one or more light sources existing in the image area of the sky in the captured image. presume. Then, based on each estimation result, the color temperature of the light source existing in the sky image area, which is within a predetermined color temperature determination range including the color temperature of the image area of the achromatic member, is determined as the final color temperature. It is assumed that In this technology, not only is it necessary to place an achromatic member at a predetermined position in the vehicle, but the achromatic member that becomes this color reference is contaminated or exposed to a local light source species. Therefore, it is impossible to accurately estimate the color temperature.

JP 2008-158958 A (paragraph number [0011-0021], FIG. 1) JP 2009-271112 A (paragraph number [0009-0021], FIG. 1)

  An object of the present invention is to provide a simple method for detecting differences in image characteristics such as hue and luminance between captured images from a plurality of cameras without requiring a color reference member attached to a vehicle. It is to provide a parking support device that corrects and generates an overhead image that does not cause a sense of incongruity.

In order to achieve the above object, a parking assistance device according to the present invention includes an image input unit that acquires images captured by a plurality of cameras that capture images of the periphery of the vehicle with different fields of view;
A specific road marking extraction unit that extracts an identifiable road element having a specific color component from the captured image;
A captured image correction unit that generates a corrected captured image by performing image correction on the correction target image based on a specific road sign extracted from the captured image;
An overhead image generation unit that generates an overhead image as a display image around the vehicle to be displayed on a monitor by synthesizing the overhead image segments for each of the cameras generated through viewpoint conversion from an image based on the captured image; The correction target image is the captured image or the overhead image segment.

  It is attached to the road area around the shoulder of the road and the parking area partitioned by the parking lot, road markings such as white lines to indicate the boundaries, lane boundary lines drawn on the road side, side road lane markings, and road surface side grooves. There are identifiable road elements with specific color components, such as groove covers. According to this configuration, a road element having such a specific color component is extracted from a captured image acquired from each camera, and the road element having the extracted specific color component is used as a color reference for image correction. Use. That is, in each captured image, image correction is performed so that the extracted road element having a specific color component becomes a predetermined color component. In the corrected captured image obtained in this way, a road element having at least a specific color component has a specific color regardless of the main light source. Therefore, when using these corrected captured images to generate a bird's-eye view image as a display image around the host vehicle and displaying it on the monitor, it has a specific color component that is most important when parking and is easily recognized by the driver. Since the road elements are displayed in the same color, that is, a specific color, in the entire overhead image, the driver can see the overhead image displayed on the monitor for parking assistance without a sense of incongruity.

  Image correction using a road element having a specific color component as a color reference by the photographed image correction unit may be performed on the original image for generating the overhead image segment, or on the generated overhead image segment You may go. Therefore, in one preferred embodiment, the overhead image generation unit generates the overhead image segment from the corrected captured image generated by the captured image correction unit. Further, in another one of the preferred embodiments, the captured image correction unit performs image correction on the overhead image segment generated by the overhead image generation unit, and from the overhead image segment subjected to the image correction, the image correction unit performs the image correction. An overhead image is generated.

  A particularly suitable image correction in the image correction using the road elements as described above as a color reference is white balance correction. Therefore, as a particularly preferred embodiment, the specific road element is a white road sign, the image correction is a white balance correction, and the photographed image correction unit is a white road sign extracted from the photographed image. A parking assist device is proposed that performs white balance correction on the captured image to generate a corrected captured image. In other words, this preferred parking assist device is characterized by an image input unit that acquires captured images from a plurality of cameras that capture the periphery of the vehicle with different field of view, and a white system that extracts a white road sign from the captured image. A road sign extraction unit, a shot image correction unit that generates a corrected shot image by performing white balance correction on the shot image based on a white road sign extracted from the shot image, and a viewpoint from the corrected shot image An overhead image generation unit that combines the overhead image segments for each camera generated through the conversion and generates an overhead image as a display image around the host vehicle displayed on the monitor.

  White road markings such as a white line for indicating the boundary and a lane boundary drawn on the road side exist in the parking area partitioned around the road shoulder and the parking lot. According to this configuration, such a white road marking is extracted from a captured image acquired from each camera, and the extracted white road marking is used as a color reference for white balance correction. That is, color correction is performed so that the extracted white road marking becomes a predetermined white color in each captured image. In the corrected captured image obtained in this way, at least the white road marking has a predetermined white color regardless of the main light source. Therefore, when a bird's-eye view image as a display image around the host vehicle is generated from these corrected captured images through viewpoint conversion and displayed on the monitor, the parking boundary line that is most important at the time of parking and is noticed by the driver is shown. A white road marking such as a white line is displayed in the same color in the entire overhead image, that is, a predetermined white color, so that the driver can see the overhead image displayed on the monitor for parking assistance without a sense of incongruity. .

  In parking lots, there is often a straight parking white line to show the boundary of the parking area as an example of a white road marking, so each captured image for generating an overhead image has such white There is a high possibility that road markings are included. However, depending on the orientation and posture of the vehicle, there is a possibility that a photograph taken by a specific camera does not include a white road marking. Therefore, in one preferred embodiment of the present invention, the captured image correction unit performs white balance correction used at the time of generating the previous corrected captured image on the captured image from which the white road marking has not been extracted. A corrected captured image is generated by performing pseudo white balance correction based on the parameter. According to this configuration, if a white road marking is not extracted from a photographed image to be subjected to white balance correction, white balance correction is performed using the white balance correction parameter used last time. Since the state of the environmental light source is unlikely to change significantly at short time intervals, if it is not more than a predetermined distance in time or distance from the previous white road marking extraction point, it is corrected based on the previous white balance correction parameter Even so, a satisfactory corrected photographed image can be obtained.

  Depending on the parking route and the vehicle posture, a state where a white road marking is not extracted from a photographed image of a specific camera may continue. In order to cope with such a situation, in one of the preferred embodiments of the present invention, the pseudo white balance correction for the captured image from which the white road marking is not extracted is performed by a camera adjacent to the camera of the captured image. Correction of the photographed image This is performed based on the white balance correction parameter used when generating the photographed image. In this configuration, images taken by adjacent cameras are likely to have the same main light source, so the white balance correction parameter used for one of the corrections is used. As a result, color misalignment or the like that causes a sense of incongruity in the mutually corrected captured images is suppressed.

  In one preferred embodiment of the present invention, a parking area setting unit is provided for setting a parking area with a parking white line drawn on a road surface as a boundary reference, and the recognition information of the parking white line is the white road. Used as a sign. This configuration uses a white line recognition technique as a white road marking in a parking assist device that supports a driving operation for parking a vehicle at a predetermined parking position, which is conventionally known. For example, in the parking assist device described in Japanese Patent Application Laid-Open No. 2008-284969, a white line indicating a parking area is image-recognized, a parking target position is determined based on the recognition result, and the parking target position and the reverse start position are determined. A guidance route is calculated based on the position of the stopped vehicle, and parking assistance is performed based on the guidance route. Thus, by incorporating the white balance correction technology according to the present invention into a parking assistance device having a parking white line recognition technology, it is possible to monitor and display a bird's-eye view image without a sense of incongruity. In other words, by extracting the parking white line from the captured image based on the information regarding the recognized parking white line and performing white balance correction using the extracted parking white line as a color reference, a bird's-eye view image without a sense of incongruity is generated. The

A basic generation process used in a preferred embodiment of the parking assistance apparatus according to the present invention to perform white balance correction on an image captured by a plurality of cameras and generate an overhead image based on the corrected captured image will be described. It is a schematic diagram. 1 is a perspective view of a vehicle in which a part of a vehicle equipped with a parking assist device according to the present invention is cut away. It is a top view which shows the imaging | photography range of a some vehicle-mounted camera. It is a functional block diagram which shows typically a part of vehicle control system containing a parking assistance control unit. It is a functional block diagram of an image processing module. It is explanatory drawing which shows typically the process of performing white balance correction | amendment and pseudo | simulation white balance correction | amendment using a white line with respect to a picked-up image. It is explanatory drawing which shows typically the process in which a bird's-eye view image is produced | generated from a correction | amendment picked-up image. It is a monitor screen figure which shows the monitor display image containing the bird's-eye view image with which the white line was emphasized.

  First, a preferred embodiment of a parking assistance device according to the present invention will be described with reference to FIG. Here, the specific road element is a white road marking, and the image correction is white balance correction. That is, a configuration is adopted in which white balance correction is performed on the captured image to generate a corrected captured image. A basic generation process for generating a bird's-eye view image based on a photographed image corrected by performing white balance correction on the photographed image based on a white road marking extracted from the photographed images by a plurality of cameras will be described. In this example, white balance correction is performed on a plurality of captured images acquired by four cameras during parking in a reverse garage, based on the white road marking for parking, and the surrounding area of the vehicle is viewed from the corrected captured images. An image (hereinafter simply referred to as an overhead image) is generated. A parking white line (hereinafter simply referred to as a white line) WM that bounds the parking area is employed as an example of a white road marking for parking. The passenger car (hereinafter referred to as the own vehicle) that is parking-supported here is equipped with four cameras each having a photographing field of view at the rear, the front, and the left and right sides. Here, (1a) acquired by each camera. , (1b), (1c), and (1d), the surrounding area of the vehicle is substantially covered with no gaps.

  When a parking area (in this example, an area partitioned by two white lines WM extending in the parking direction) is determined and reverse parking is started, the captured images (1a), (1b), (1c) and (1d) are acquired (# 01). From the acquired captured image, white line WM detection processing is performed using a known white line detection algorithm. When the white line WM is detected, a white line region surrounded by the boundary line of the white line WM is extracted, and white line information including an average pixel value (for example, R, G, B value) of the region is generated (# 02). When the white line information is generated, a correction parameter is generated based on the content of the white line information and the image characteristics (ka: average pixel value, luminance histogram, etc.) of the corresponding captured image (# 03). This correction parameter means a parameter group in which the white line WM of the photographed image to be processed is converted into a predetermined white color, and is a white balance correction parameter (white balance correction curve: Pa = f (ka)). is there. These white balance correction parameters (Pa = f (ka); Pb = f (kb); Pc = f (kc); Pd = f (kd)) are all taken images (1a), (1b), (1c). ), (1d). Subsequently, by applying this white balance correction parameter to the entire photographed image, white balance correction is executed on the four photographed images, and four corrected photographed images are generated (# 04). In these four corrected photographed images, at least the color of the white line WM is unified, and color fogging in other image areas is also suppressed, so that the difference in mutual hue is improved. Next, by converting the viewpoint of each corrected photographed image, an overhead image segment for each camera is generated, and these overhead image segments are combined to generate an overhead image (# 05). The generated overhead image is displayed on a monitor together with, for example, a back monitor image using the image captured by the back camera as it is (# 06).

  In the embodiment described with reference to FIG. 1, the procedure for generating the overhead image segment from the corrected captured image that has been white balance corrected using the white road marking for parking has been shown. Alternatively, a procedure may be employed in which an overhead image segment is generated and white balance correction is performed on the overhead image segment using a white road marking for parking.

Hereinafter, specific examples of embodiments of the present invention will be described with reference to the drawings. The parking assist device according to the present embodiment monitors a bird's-eye view image generated from a captured image of a camera that captures the parking direction and the left-right direction among the front camera, left-right side camera, and rear camera provided in the host vehicle. To display.
As shown in FIGS. 2 and 3, the vehicle is configured such that the left and right front wheels 11a and 11b are steered wheels and the left and right rear wheels 12a and 12b are non-steered wheels. A rear camera 1 a is provided at the rear of the vehicle, that is, the back door 91. A left side camera 1b is provided below the left side mirror 92 provided on the left front door, and a right side camera 1c is provided below the right side mirror 93 provided on the right front door. A front camera 1d is provided at the front of the vehicle. In the following description, these cameras 1a to 1d will be collectively referred to as a camera 1 (vehicle camera) as appropriate.

  The camera 1 uses an image sensor such as a charge coupled device (CCD) or a CMOS image sensor (CIS) to take a two-dimensional image of 15 to 30 frames per second in time series, and digitally converts the captured image in real time. It is a digital camera that outputs. The camera 1 includes a wide angle lens. In particular, in the present embodiment, a viewing angle of approximately 180 ° is ensured in the horizontal direction. The rear camera 1a and the front camera 1d are installed in a vehicle having a depression angle of about 30 degrees on the optical axis, and can capture an area of about 8 m from the vehicle. The left side camera 1b and the right side camera 1c are installed at the bottoms of the side mirrors 92 and 93 with the optical axis facing downward, and part of the side surface of the vehicle and the road surface (ground) are within the field of view. In the parking assistance, an image captured by the camera 1 or an overhead image generated using the captured image is displayed on the monitor 21. In addition, voice guidance based on a parking route from a parking departure point to a target parking area is also emitted from the speaker 22. Confirmation of the parking area and other operation inputs are performed through the touch panel 21T attached to the monitor 21.

  Inside the vehicle, a parking support control unit (hereinafter simply referred to as ECU) 20 that is the core of the parking support device of the present invention is arranged. As shown in FIG. 4, the ECU 20 includes a sensor input interface 23, a communication interface 70, and the like as input / output interfaces for inputting / outputting information, and a microprocessor for processing information obtained via the input / output interface, A DSP (digital signal processor) is provided. Also, some or all of the input / output interfaces may be included in such a processor.

The sensor input interface 23 is connected to a vehicle state detection sensor group for detecting a driving operation and a moving state. The vehicle state detection sensor group includes a steering sensor 24 that measures the steering operation direction (steering direction) and the operation amount (steering amount), a shift position sensor 25 that determines the shift position of the shift lever, and an accelerator pedal operation amount. An accelerator sensor 26 for detecting the brake pedal, a brake sensor 27 for detecting the operation amount of the brake pedal, a distance sensor 28 for detecting the travel distance of the host vehicle, and the like.
The communication interface 70 employs an in-vehicle LAN, and not only the touch panel 21T and the monitor 21, but also control units such as a power steering unit PS, a transmission mechanism T, and a brake device BK are connected so as to be able to transmit data.

  The ECU 20 includes a display control unit 71 and a sound processing module 72 as a notification output function unit. Captured image information including various image information for GUI and a notification message for vehicle guidance is displayed on the monitor 21 via the display control unit 71. A voice guide for vehicle guidance, an emergency warning sound, and the like generated by the voice processing module 72 are output from the speaker 22. Further, the ECU 20 includes a white road marking recognition unit 30, a parking area setting unit 31, a target parking position calculation unit 32, a parking route generation unit 33, a guidance control unit 34, a position information calculation unit 35, An image processing module 50 is provided.

  Here, the white road marking recognizing unit 30 is configured as a parking white line recognizing unit that recognizes a white line that determines a section of the parking area. This white line recognition is performed, for example, by performing straight line fitting using Hough transform or the like on edge pixels obtained by applying a differential filter to a captured image temporarily stored in the image processing module 50. Such a white line recognition algorithm is known per se, and is described in detail in, for example, Japanese Patent Application Laid-Open No. 2006-175918 and Japanese Patent Application Laid-Open No. 2008-284969.

The parking area setting unit 31 sets a parking area for parking the vehicle from the position information of the white line recognized by the white road marking recognition unit 30. In addition, you may employ | adopt the system which sets the setting of this parking area by manual operation through the picked-up image displayed on the monitor 21. FIG.
The target parking position calculation unit 32 sets the parking completion position as the target parking position, which is the arrival point of parking travel in the parking area set by the parking area setting unit 31, and the dimension of the parking area and the size of the own vehicle. . When the own vehicle stops at the target parking position, the own vehicle is parked at an appropriate position (for example, the middle) in the parking area.

  The parking route generation unit 33 generates a parking route. This parking route is a route connecting from the parking departure point where parking assistance is started to the parking completion position which is the parking target position, and may include a straight line partially or as a whole as a straight line. Can be substantially represented by one or more curves. Incidentally, the generation algorithm of the parking route has been conventionally known, and reference can be made to Japanese Patent Laid-Open No. 2003-237511, Japanese Patent Laid-Open No. 2008-284969, and the like.

  The position information calculation unit 35 acquires the current vehicle position necessary for guidance of the host vehicle and the position of the target parking area with respect to the host vehicle when the host vehicle moves. That is, the position information calculation unit 35 detects the relative position relationship between the vehicle position detection process for detecting the position information of the own vehicle that changes with the movement of the own vehicle and the target parking area that changes with the movement of the own vehicle. The parking area relative position calculation process for calculating These processes are performed based on the captured image acquired by the camera 1, the vehicle movement amount acquired by the distance sensor 28, and the steering amount of the steering wheel measured by the steering sensor 24.

  The guidance control unit 34 guides parking based on the direct parking route generated by the parking route generation unit 33. At that time, the position information from the position information calculation unit 35 is referred to. The guidance control unit 34 can realize control for driving the host vehicle by guidance control along the parking route while referring to the location information from the location information calculation unit 35. However, automatic steering in which the guidance control unit 34 controls the power steering unit PS, the transmission mechanism T, and the brake device BK is limited to backward traveling only, and the forward steering is partially manual steering. It may be incorporated. Since the guidance information by the guidance control unit 34 is sent to the image processing module 50 and the sound processing module 72, the steering direction and the steering amount are displayed on the monitor 21 as required, or the steering direction and the steering amount are displayed on the speaker. 22 is output. In any case, since the bird's-eye view image generated by the image processing module 50 is displayed on the monitor 21, the driver can easily understand the positional relationship of the vehicle in the parking process.

  FIG. 5 shows a functional block diagram of the image processing module 50 of the ECU 20. The image processing module 50 includes a mechanism for matching at least the color of the white line WM to a predetermined color so as to give a sense of unity to the captured images acquired by the plurality of cameras 1 that capture the periphery of the vehicle, And a function of generating a bird's-eye view image by viewpoint conversion as a viewpoint.

  The image processing module 50 includes an image input unit 51, a captured image memory 52, a white road marking extraction unit 53, a captured image correction unit 54, an image generation unit 60, and an image composition unit 67. The image input unit 51 develops the captured image acquired by each camera 1 in the captured image memory 52. The white road marking extraction unit 53 here converts the white line WM, which is a parking lot line, from the corresponding captured image based on the white line recognition information sent from the white road marking recognition unit 30 to the image area of the white line WM. The image feature amount such as the pixel value is extracted.

The captured image correction unit 54 includes a correction parameter calculation unit 55, a white balance correction unit 56, and a pseudo white balance correction unit 57. Next, the correction function by the photographed image correction unit 54 will be described with reference to FIG. The correction parameter calculation unit 55 includes the image feature amount related to the white line WM extracted by the white road marking extraction unit 53, and the processing target captured images ((1a), (1b), ( From the correction parameter calculation factor groups ka, kb, kc, and kd that are image feature amounts of any of 1c) and (1d)), the color of the white line WM recognized in the captured image to be processed is set to a predetermined color (for example, A white balance correction parameter (hereinafter simply referred to as a correction parameter) for obtaining a preset white color) is calculated. If the function for deriving this correction parameter is f and the correction parameters for each captured image are Pa, Pb, Pc, Pd,
Pa = f (ka), Pb = f (kb), Pc = f (kc), Pd = f (kd),
It can be expressed as. The correction parameter is generally expressed as a matrix, but functionally, it is like a conversion curve for converting an input value into a predetermined output value.

The white balance correction unit 56 performs white balance correction on the corresponding captured image using each correction parameter Pa, Pb, Pc, Pd, that is, [corrected captured image (1a)] = [captured image (1a)]. * Matrix operation like [Pa] is performed to generate each corrected photographed image.

  In FIG. 6, when the white line WM is not recognized as in the captured image (1c) or the captured image (1d), the pseudo white balance correction unit 57 performs pseudo white balance correction as described below. It has a function to be applied to a captured image in which WM is not recognized. Two examples of this pseudo white balance correction are shown here.

The first pseudo white balance correction is based on the white line WM, if there is a white line WM recognized in the previous captured image in the current parking process by the camera 1 that acquired the captured image in which the white line WM was not recognized. This is white balance correction performed using correction parameters. In other words, the correction is performed using the correction parameter used when the time or travel distance goes back by a predetermined value. The second pseudo white balance correction is based on the white line WM when the white line WM is recognized in a captured image of an adjacent camera 1 other than the camera 1 that has acquired the captured image in which the white line WM was not recognized. This is white balance correction performed based on the correction parameters. For example, in the example of FIG. 6, since the white line WM is recognized in the captured image (1b) by the camera 1b adjacent to the camera 1d that acquires the captured image (1d), the correction parameter Pb based on the white line WM is used. That is, using the correction parameter derived by Pd = g (Pb), white balance correction of the captured image (1d) is performed. Similarly, the photographic image (1c) white balance correction is performed using the correction parameter derived by Pc = g (Pa). Further, although different from the illustrated example, when the white line WM is recognized in the captured images (1b) and (1c) by the cameras 1b and 1c adjacent to the camera 1d that acquires the captured image (1d), It is also possible to derive the correction parameter Pd for the captured image (1d) from the correction parameter Pd based on the white line WM. This derivation is expressed as Pd = h (Pb, Pc). For example, a function for obtaining an intermediate value can be adopted as the function h.
As a selection criterion for the first or second pseudo white balance correction, the first pseudo white balance correction is prioritized on the condition that the correction parameter Pd generated within a predetermined travel distance exists. Is preferred.

The image generation unit 60 includes a normal image generation unit 61, a surrounding bird's-eye view image generation unit 62, a mapping table 63, an enhanced sign image generation unit 64, a host vehicle image generation unit 65, and a notification image generation unit 66.
The normal image generation unit 61 adjusts the captured image to an image quality suitable for monitor display as a vehicle peripheral image as it is. The vehicle periphery image displayed on the monitor is independent for each camera, and the monitor display can be arbitrarily selected.

  As schematically shown in FIG. 7, the peripheral bird's-eye view image generation unit 62 uses the viewpoint conversion table prepared for each virtual viewpoint in the mapping table 63 with the corrected captured image corrected by the captured image correction unit 54. Is used to generate an overhead image segment and send it to the image composition unit 67. The virtual viewpoint position used at that time may be set in advance or may be set arbitrarily. Although the viewpoint conversion table of the mapping table 63 can be constructed in various forms, it is constructed as a table describing the correspondence between the pixel data of the captured image and the pixel data of the overhead image segment, and captures one frame. Conveniently, the destination pixel coordinates in the overhead image segment are described in each pixel of the image.

Here, the emphasized sign image generating unit 64 generates a white line image visually enhanced in color and / or shape of the white line WM extracted by the white road sign extracting unit 53, and the white line image is generated. Are sent to the image composition unit 67 together with the coordinate position in the overhead image.
The own vehicle image generation unit 65 prepares an image of the own vehicle arranged at the center of the overhead image and sends it to the image composition unit 67. When a message or the like to be given to the driver at the time of parking assistance is generated, the notification image generation unit 66 converts this into an image and sends it to the image composition unit 67.

  The image synthesizing unit 66 adds the emphasis indication image from the emphasis indication image generation unit 64 and the own vehicle image from the own vehicle image generation unit 65 to the overhead view image generated by synthesizing the overhead view image segments from the peripheral overhead view image generation unit 62. Then, a notification image from the notification image generation unit 65 is synthesized to generate a monitor display overhead image (display image). In this case, as the monitor display image, as illustrated in FIG. 8, it is convenient to divide and arrange not only the overhead image but also the back monitor image and the like for monitor output.

Next, a series of operations in the parking assistance control by the parking assistance device configured as described above will be described below.
(1) When performing reverse parking, the parking area is stopped at a position where the parking area is within the field of view of the rear camera 1a, and parking assistance is activated.
(2) Based on the photographed image from the rear camera 1a, the white road marking recognition unit 30 performs white line recognition and generates recognition information regarding the recognized white line WM.
(3) Based on this recognition information, the parking area setting unit 31 sets the parking area, and the target parking position calculation unit 32 calculates the target parking position.
(4) While the parked vehicle position is set as the parking departure position, the parking route generation unit 33 generates a parking route between the parking departure position and the parking target position.
(5) In the image processing module 50, captured images from the four cameras 1 are acquired, and white balance correction is performed on each captured image in the procedure shown in FIG.
(6) Using the corrected captured image, a bird's-eye view image is generated according to the procedure shown in FIG. 7, and a monitor display image as shown in FIG.
(7) When vehicle guidance by the guidance control unit 34 starts, a notification message such as a vehicle guidance message as parking assistance is also displayed on the monitor 21 together with an overhead image.
(8) Each time a predetermined distance is traveled, it is checked whether or not the vehicle has reached the parking target position. If the vehicle has not reached the parking target position, a bird's-eye view image with a new captured image is generated. Then, the vehicle guidance continues while displaying this on the monitor.
(9) If the host vehicle reaches the parking target position, the host vehicle stops, and the parking support control ends.

[Modification]
(1) In the above-described embodiment, the captured image is handled as a color image, white balance correction parameters are calculated using the recognized white line as a reference, white balance correction is performed using the correction parameters, and the corrected captured image is corrected. A bird's-eye view image was generated. Instead, a bird's-eye view image may be generated from a monochrome captured image obtained by performing gray scale conversion on the corrected captured image and used as a monitor image. Furthermore, this white balance correction technique is directly applied to monochrome image processing, a correction parameter is calculated from a monochrome captured image, image correction is performed using the correction parameter, and an overhead image is generated from the corrected captured image. May be. That is, the white balance correction here should be interpreted in a broad sense, and can be applied not only to a color photographed image but also to a monochrome photographed image, for example.

(2) In the embodiment described above, the white line (parking) that bounds the parking area is used as the white road marking. However, other road markings such as “IN (entrance)” and “OUT (exit)” It is possible to handle character markings such as

[Another embodiment]
In the embodiment described above, the specific road element is a white road marking, and the image correction is white balance correction. However, the present invention is not limited to such specific road elements or image correction. For example, roads having specific color components that can be identified as specific road elements, such as road signs such as road lane markings and side road lane markings, curbs laid on the road surface, and groove covers mounted on the road surface side grooves Any element can be used as a color reference. In addition to the white balance correction, the photographed image correction unit 54 can also be configured to perform image correction such as color matching (tone adjustment), luminance adjustment, and gamma adjustment.

  INDUSTRIAL APPLICABILITY The present invention can be used in a parking assistance device that supports a driver's driving operation by displaying a bird's-eye view image for allowing the driver to recognize the situation around the host vehicle during parking.

1: Camera 30: White road marking recognition unit 50: Input image processing module 51: Image input unit 53: White road marking extraction unit 54: Captured image correction unit 55: Correction parameter calculation unit 56: White balance correction unit 57: Pseudo white balance correction unit 60: image generation unit 62: peripheral overhead image generation unit 63: mapping table 64: emphasis marking image generation unit WM: white line (parking white line, white road marking)

Claims (7)

An image input unit for acquiring images taken by a plurality of cameras that shoot around the own vehicle in different fields of view;
A specific road marking extraction unit that extracts an identifiable road element having a specific color component from the captured image;
A captured image correction unit that generates a corrected captured image by performing image correction on the correction target image based on a specific road element extracted from the captured image;
An overhead image generation unit that synthesizes an overhead image segment for each camera generated through viewpoint conversion from an image based on the captured image and generates an overhead image as a display image around the host vehicle displayed on a monitor; The parking support device in which the correction target image is the captured image or the overhead image segment.   The parking assistance device according to claim 1, wherein the overhead image generation unit generates the overhead image segment from the corrected captured image generated by the captured image correction unit.   The parking according to claim 1, wherein the captured image correction unit performs image correction on the overhead image segment generated by the overhead image generation unit, and the overhead image is generated from the image-corrected overhead image segment. Support device. The specific road element is a white road marking, and the image correction is a white balance correction;
The photographed image correcting unit according to any one of claims 1 to 3, wherein the photographed image correcting unit generates a corrected photographed image by performing white balance correction on the photographed image based on a white road sign extracted from the photographed image. The parking assistance device described.   The photographed image correction unit performs a corrected white balance correction on a photographed image from which the white road marking has not been extracted based on a white balance correction parameter used when generating a previous corrected photographed image. The parking assistance device according to claim 4 which generates an image.   The pseudo white balance correction for the captured image from which the white road marking is not extracted is performed based on the white balance correction parameter used when the corrected captured image is generated by the camera adjacent to the camera of the captured image. The parking assistance device according to claim 4 or 5.   The parking area setting part which sets a parking area on the basis of the parking white line drawn on the road surface as a boundary reference is provided, and the recognition information of the parking white line is used as the white road marking. The parking assistance device according to claim 1.

Images