JP6519408B2 - Display device for vehicle and display method for vehicle - Google Patents

Description

  The present invention relates to display technology for vehicles, and more particularly to a display apparatus for vehicles displaying an image and a display method for vehicles.

  Cameras may be installed at multiple locations on the vehicle. At that time, the overhead image is generated by subjecting the photographed image to viewpoint conversion and combining (for example, see Patent Document 1).

Unexamined-Japanese-Patent No. 2012-217000

  In the viewpoint conversion, the distance between objects having a difference in position in the vertical direction can not be accurately represented, which may cause discomfort or the distance to the object may not be accurately grasped. For example, in a parking lot where a plurality of vehicles are lined up and parked, when parking is performed in a parking section by retreating, a frame line is displayed in the viewpoint converted image even if there is a parking frame line right under the end of the rear vehicle. It seems that there is a distance to the rear vehicle. In such a case, if the frame line immediately below the end of the rear vehicle is a frame line indicating the parking range of the vehicle, the distance to the vehicle behind can not be grasped appropriately.

  The present invention has been made in view of these circumstances, and it is an object of the present invention to provide a technique for accurately reflecting the distance to an object when performing viewpoint conversion.

  In order to solve the above problems, a display apparatus for a vehicle according to an aspect of the present invention includes a distance between an image acquisition unit that acquires an image obtained by imaging the periphery of the vehicle and an object imaged in the image acquired by the image acquisition unit. Image processing for deleting from the image a pattern included in the image acquired by the image acquisition unit and being located below the object according to the distance acquisition unit acquiring the image and the distance acquired by the distance acquisition unit Display processing for causing the display unit to display an image converted by the viewpoint conversion processing unit, and a viewpoint conversion processing unit converting the viewpoint as seen from above the vehicle with respect to the image in which the pattern is erased in the image processing unit And a unit.

  Another aspect of the present invention is a display method for a vehicle. This method includes the steps of acquiring an image obtained by imaging the periphery of a vehicle, acquiring a distance to an object imaged in the acquired image, and a pattern included in the acquired image according to the acquired distance. And removing the pattern located below the object from the image, converting the viewpoint of the image from which the pattern has been removed as viewed from above the vehicle, and converting the converted image to the display unit And displaying.

  Note that arbitrary combinations of the above-described components, and conversions of the expression of the present invention among methods, apparatuses, systems, recording media, computer programs and the like are also effective as aspects of the present invention.

  According to the present invention, when performing viewpoint conversion, the distance to an object can be accurately reflected.

It is a perspective view showing the imaging range formed around the vehicles concerning the example of the present invention. FIGS. 2 (a) and 2 (b) are diagrams showing an outline of processing in the comparison object of the embodiment of the present invention. It is a figure showing composition of a display for vehicles concerning an example of the present invention. It is a figure which shows the image imaged in the back imaging part of FIG. FIGS. 5 (a) and 5 (b) are diagrams showing an outline of processing in the vehicle display device of FIG. FIGS. 6 (a) and 6 (b) are diagrams showing an outline of another process in the vehicle display device of FIG. It is a flowchart which shows the display procedure by the display apparatus for vehicles of FIG. It is a flowchart which shows another display procedure by the display apparatus for vehicles of FIG.

  Before the present invention is specifically described, the premise is first described. The embodiment of the present invention relates to a display apparatus for a vehicle that generates a bird's-eye view image by performing viewpoint conversion on images captured by a plurality of imaging units installed in a vehicle, and displays the generated bird's-eye view image. When parking in a parking area by moving backwards in a parking lot where a plurality of vehicles are parked side by side, the parking frame line drawn immediately below the end of another vehicle behind should be invisible to the other vehicle when viewed from above It is. However, in the overhead image generated by the viewpoint conversion, the parking frame line may be located closer to the main vehicle without being hidden by other vehicles. Therefore, the driver may look further away from the parking frame line as if there is another vehicle behind, and the distance to the other vehicle behind can not be properly grasped.

  In order to cope with this, the display apparatus for a vehicle according to the present embodiment detects an object such as another vehicle from a vehicle peripheral image captured by a plurality of imaging units in which the vehicle is installed, and detects up to the detected object. Get the distance. Further, when the distance is shorter than the threshold value, the vehicle display device erases the parking frame line located below the object of the other vehicle from the image. Furthermore, the display apparatus for vehicles produces | generates a bird's-eye view image by performing viewpoint conversion with respect to the image which eliminated the parking frame line, and displays the produced | generated bird's-eye view image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The specific numerical values and the like shown in the examples are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the specification and the drawings, elements having substantially the same functions and configurations will be denoted by the same reference numerals to omit repeated description, and elements not directly related to the present invention will not be illustrated. .

  FIG. 1 is a perspective view showing an imaging range formed around a vehicle 100 according to an embodiment of the present invention. A front imaging unit 12 is installed at a front portion of the vehicle 100, for example, a bumper, a bonnet or the like. The front imaging unit 12 forms a front imaging area 20 so as to move forward from the front imaging unit 12 and captures an image in the front imaging area 20. The left side imaging unit 14 is installed on the left side portion of the vehicle, for example, the lower portion of the left side door mirror or the like. The left side imaging unit 14 forms the left side imaging region 22 so as to be directed leftward from the left side imaging unit 14 and captures an image in the left side imaging region 22.

  A rear imaging unit 16 is installed at a rear portion of the vehicle, for example, a bumper, a trunk or the like. The rear imaging unit 16 forms a rear imaging area 24 so as to face the rear from the rear imaging unit 16 and captures an image in the rear imaging area 24. A right side imaging unit 18 (not shown) is installed on the right side of the vehicle so as to be symmetrical with the left side imaging unit 14. The right side imaging unit 18 forms the right side imaging region 26 so as to be directed rightward from the right side imaging unit 18 and captures an image in the right side imaging region 26. The front imaging unit 12, the left side imaging unit 14, the rear imaging unit 16, and the right side imaging unit 18 are used as the imaging unit 10, and the periphery of the vehicle 100 is imaged by an image captured by the imaging unit 10.

  FIGS. 2 (a)-(b) show an outline of processing in the comparison object of the embodiment of the present invention. In the comparison target, as shown in FIG. 1, it is assumed that an image is captured by the imaging unit 10 installed in the vehicle 100. On the other hand, in the comparison object, the processing for generating a bird's-eye view image by viewpoint conversion on the image corresponds to the processing performed so far. FIG. 2A shows a case where the vehicle 100 of FIG. 1 is viewed from the left side. As described above, the rear imaging unit 16 is installed at the rear portion of the vehicle 100. In the rear of the vehicle 100, another vehicle 110 is parked or stopped. Furthermore, a frame line 30 is drawn on the ground, and the frame line 30 exists below the front end of the other vehicle 110. Therefore, when the vehicle 100 and the other vehicle 110 are viewed from above, the frame line 30 is hidden by the other vehicle 110. On the other hand, since the rear imaging unit 16 has the imaging range 32, the image captured by the rear imaging unit 16 includes the front end portion of the other vehicle 110 and the frame line 30.

  FIG. 2 (b) shows a bird's-eye view image 48 generated by performing viewpoint conversion on the image captured by the imaging unit 10. As described above, in the processing for generating the overhead image 48, the processing that has been performed up to this point is executed. The vehicle 100 is disposed in the central portion of the overhead image 48. A front image 40 is disposed in front of the vehicle 100, a left side image 42 is disposed on the left side of the vehicle 100, a rear image 44 is disposed on the rear of the vehicle 100, and a right side An image 46 is placed. In particular, the rear image 44 is generated based on an image obtained by imaging the imaging range 32 in FIG. Therefore, in the rear image 44, the other vehicle 110 is disposed behind the vehicle 100, and the frame 30 is disposed between the vehicle 100 and the other vehicle 110. That is, the frame line 30 which should originally be hidden by the other vehicle 110 does not hide by the other vehicle 110, and the overhead image 48 and the actual situation are different.

  FIG. 3 shows the configuration of a display device 50 for a vehicle according to an embodiment of the present invention. The vehicular display device 50 is connected to the front imaging unit 12, the left side imaging unit 14, the rear imaging unit 16, and the right side imaging unit 18 which constitute the imaging unit 10. Vehicle display device 50 includes an image acquisition unit 52, a distance acquisition unit 54, an image processing unit 56, a viewpoint conversion processing unit 58, a display processing unit 60, and a display unit 62.

  The front imaging unit 12, the left side imaging unit 14, the rear imaging unit 16, and the right side imaging unit 18 capture an image as described above. The image is a moving image, but may be a still image. The front imaging unit 12, the left side imaging unit 14, the rear imaging unit 16, and the right side imaging unit 18 output an image to the image acquisition unit 52.

  The image acquisition unit 52 inputs an image from each of the front imaging unit 12, the left side imaging unit 14, the rear imaging unit 16, and the right side imaging unit 18. That is, the image acquisition unit 52 acquires an image obtained by imaging the periphery of the vehicle 100. FIG. 4 illustrates an example of an image captured by the rear imaging unit 16. The example of the image shown in FIG. 4 does not show the distortion of the image due to the fact that each of the imaging units 10 is a camera capable of wide-angle shooting, for ease of explanation. Here, as shown in FIG. 2A, it is assumed that another vehicle 110 is parked at the rear of the vehicle 100 and a frame line 30 is drawn on the ground below the other vehicle 110. The image includes the other vehicle 110 and the frame 30. Return to FIG. The image acquisition unit 52 outputs the acquired image to the distance acquisition unit 54.

  The distance acquisition unit 54 inputs the image from the image acquisition unit 52. In the following, although processing on an image captured by the rear imaging unit 16 will be described in order to clarify the description, similar processing may be performed on other images. The distance acquisition unit 54 executes edge detection processing on the image. The edge detection process is a type of feature detection or feature extraction, and is an algorithm for identifying a portion where the brightness of an image is discontinuously changing. To detect. Further, the distance acquisition unit 54 also detects a pattern included in the image, for example, the frame line 30 by edge detection processing. In addition, since the edge detection processing is a known technique, the detailed description is omitted here. The frame line 30 referred to here is generally a white line drawn on a road surface, and a known white line detection technique based on a captured image can be applied. Moreover, it is the same as when the frame 30 is not the frame but the curb.

  The distance acquisition unit 54 acquires the distance to the detected object. An object which is a target for which the distance acquisition unit 54 acquires a distance is, for example, the other vehicle 110. For distance measurement with an object, known techniques may be used, and for example, stereo distance measurement and a twin-lens camera are used. This calculates the parallax which arises between a plurality of optical systems, and measures the distance to the object based on the parallax. In this case, the rear imaging unit 16 is provided with a plurality of optical systems. In addition, a distance sensor may be used for distance measurement with an object. A distance sensor (not shown) is provided in the vicinity of the rear imaging unit 16. The distance sensor irradiates the object with infrared light, laser light or the like to the object, and then receives the reflected light to obtain the imaging position information. Calculate the distance from A millimeter wave or the like may be used as the distance sensor.

  Also, vector detection processing may be used for distance measurement with an object. The vector detection process detects the specific point in the image and measures the distance to the object by calculating the amount of vector movement of the specific point. 5 (a) and 5 (b) show an outline of processing in the display device 50 for a vehicle. FIG. 5 (a) is shown similarly to FIG. 2 (a). The distance acquisition unit 54 measures the distance "d1". FIG. 5 (b) will be described later and returns to FIG. 3. The distance acquisition unit 54 outputs the acquired distance and the image to the image processing unit 56. The distance acquisition unit 54 also outputs information on the detected object and pattern to the image processing unit 56.

  The image processing unit 56 inputs the distance and the image from the distance acquisition unit 54. The image processing unit 56 also receives, from the distance acquisition unit 54, information on objects and patterns. The distance acquisition unit 54 determines, based on the information on the object and the pattern, whether or not the pattern exists below the object. In the case of FIG. 4, the frame line 30 which is a pattern exists below the other vehicle 110 which is an object. That is, the distance acquiring unit 54 determines that the pattern exists below the object when the pattern exists below the object on the image. When the pattern exists below the object, the image processing unit 56 executes the following processing.

  The image processing unit 56 holds the threshold in advance, and compares the input distance with the threshold. If the distance is shorter than the threshold value, the image processing unit 56 deletes from the image a pattern included in the image and located below the object. In the case of FIG. 4, the image processing unit 56 deletes the frame line 30 drawn on the ground from the image. Further, if the distance d1 shown in FIG. 5A is shorter than the threshold value, the image processing unit 56 deletes the frame line 30. That is, the distance acquisition unit 54 deletes the pattern from the image according to the acquired distance. The image processing unit 56 outputs the image to the viewpoint conversion processing unit 58.

  The viewpoint conversion processing unit 58 receives an image from the image processing unit 56. The viewpoint conversion processing unit 58 converts the viewpoint of the image as viewed from above the vehicle 100. Although a known technique may be used for the conversion, for example, each pixel of the image is projected on a three-dimensional curved surface in a virtual three-dimensional space, and the three-dimensional image is displayed according to the virtual viewpoint from above the vehicle 100 Cut out the required area of the curved surface. The cut out area corresponds to an image obtained by converting the viewpoint. An example of an image obtained by converting the viewpoint is shown in FIG. The vehicle 100 is disposed at the central portion of the overhead image 78 in FIG. 5 (b). A front image 70 is disposed in front of the vehicle 100, a left side image 72 is disposed on the left side of the vehicle 100, a rear image 74 is disposed on the rear of the vehicle 100, and a right side An image 76 is arranged. In particular, the rear image 74 does not include the border 30 which has already been erased.

  6 (a) and 6 (b) show an outline of another process in the display device 50 for a vehicle. FIG. 6 (a) is shown similarly to FIG. 5 (a), but in these, the distance between the vehicle 100 and the other vehicle 110 is different, and the frame line 30 is below the front end of the other vehicle 110. Does not exist. In FIG. 6A, the vehicle 100 and the other vehicle 110 are separated by a distance d2. Here, the distance d2 is longer than the distance d1 and is equal to or more than the threshold value in the image processing unit 56. Therefore, the frame line 30 is not deleted in the image processing unit 56. The overhead image 78 in FIG. 6B is shown in the same manner as FIG. 5B, but in the rear image 74, a frame line 30 is included between the vehicle 100 and the other vehicle 110. Return to FIG. The viewpoint conversion processing unit 58 outputs the overhead image 78 which is the converted image to the display processing unit 60.

  The display processing unit 60 inputs the overhead image 78 from the viewpoint conversion processing unit 58. The display processing unit 60 causes the display unit 62 to display the overhead image 78. The display unit 62 displays a bird's-eye view image 78 as shown in FIGS. 5 (b) and 6 (b).

  When the image processing unit 56 determines that the pattern is present below the object, the image processing unit 56 may execute processing different from the processing described above. The image processing unit 56 specifies the number of pixels between the object and the pattern based on the information on the object and the pattern. Further, the distance acquisition unit 54 specifies the angle of view of the image based on the input distance. Furthermore, the distance acquisition unit 54 stores in advance the relationship between the number of pixels and the actual distance for each angle of view, and based on the specified number of pixels and the angle of view, the distance between the object and the pattern Derive The derived distance is shown as distance d3 in FIG. 5 (a) and as distance d4 in FIG. 6 (a).

  The image processing unit 56 holds in advance a threshold different from the above-described threshold, and compares the derived distance with the threshold. If the distance is shorter than the threshold value, the image processing unit 56 deletes from the image a pattern included in the image and located below the object. The distance d3 in FIG. 5A is shorter than the threshold, and the distance d4 in FIG. 6A is equal to or greater than the threshold. Therefore, as in the above, the image processing unit 56 erases the frame 30 with respect to FIG. 5A and does not erase the frame 30 with respect to FIG. 6A. The subsequent processing is the same as before.

  Although two types of processing in the image processing unit 56 have been described, the threshold value is a fixed value in both cases. On the other hand, the threshold may be set according to the position where the rear imaging unit 16 and the like are attached. That is, the image processing unit 56 may set the threshold value in accordance with the height of the rear imaging unit 16 or the like from the ground. As the height of the rear imaging unit 16 or the like increases, the image captured by the rear imaging unit 16 or the like becomes closer to the image viewed from above the vehicle 100. In that case, although the frame line 30 actually exists under the other vehicle 110, when the viewpoint conversion is performed, a situation where the frame line 30 is shown between the vehicle 100 and the other vehicle 110 occurs. It becomes difficult to do. Therefore, the threshold value should be adjusted so that the pattern of the frame 30 or the like is not easily erased. Taking these into consideration, the distance acquiring unit 54 sets a threshold value that becomes shorter as the height of the rear imaging unit 16 or the like becomes higher. For example, when attaching the rear imaging unit 16 or the like to the vehicle 100, a threshold is set from the outside.

  In terms of hardware, this configuration can be realized with the CPU, memory, or other LSI of any computer, and with software, it can be realized by a program loaded into the memory, etc. Are drawing functional blocks. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The operation of the vehicular display device 50 having the above configuration will be described. FIG. 7 is a flowchart showing a display procedure by the display device 50 for a vehicle. The distance acquisition unit 54 acquires the distance to the other vehicle 110 behind (S10). If the distance is shorter than the threshold (Y in S12), the image processing unit 56 erases the frame line 30 (S14). On the other hand, if the distance is not shorter than the threshold (N in S12), the process proceeds to step S16. The viewpoint conversion processing unit 58 converts the viewpoint of the image (S16). The display processing unit 60 causes the display unit 62 to display an image (S18).

  FIG. 8 is a flowchart showing another display procedure by the vehicular display device 50. As shown in FIG. The distance acquisition unit 54 acquires the distance to the other vehicle 110 behind (S50). The image processing unit 56 derives the distance between the rear other vehicle 110 and the frame 30 (S52). If the distance is shorter than the threshold (Y in S54), the image processing unit 56 erases the frame line 30 (S56). On the other hand, if the distance is not shorter than the threshold (N in S54), the process proceeds to step S58. The viewpoint conversion processing unit 58 converts the viewpoint of the image (S58). The display processing unit 60 causes the display unit 62 to display an image (S60).

  The process of erasing the frame line 30 in step S14 and step S56 will be described. For example, when the frame line 30 detected by the distance acquisition unit 54 is a white line indicating a section of a parking lot, the area adjacent to the white line is copied by filling the white line portion with the same color as the peripheral color of the white line. Processing such as attaching on a white line or drawing a line covering the white line with a color such as black on the white line is applied. In this case, a frame line 30 perpendicular or substantially perpendicular to each imaging direction of the imaging unit 10 is to be erased. Alternatively, the frame line 30 parallel to or substantially parallel to the side of the other vehicle 110 facing the imaging unit 10 for capturing the other vehicle 110 is to be erased. The frame 30 shown in FIG. 2 (b) satisfies all the conditions.

  According to the present embodiment, the frame line located below the other vehicle is erased from the image according to the distance to the other vehicle to generate the overhead image, so the frame line to be hidden by the other vehicle is displayed in the overhead image It can avoid being done. In addition, it is possible to avoid displaying a frame line that should be hidden by another vehicle in the overhead image, so the actual situation and the overhead image can be made closer. In addition, since the actual situation and the overhead view image are close, when performing viewpoint conversion, the distance to the object can be accurately reflected. Further, since the distance to another vehicle and the threshold value are simply compared, the processing can be simplified. In addition, since the distance between the other vehicle and the frame is compared with the threshold value, the processing accuracy can be improved. In addition, since the threshold is set according to the height of the imaging unit from the ground, the processing accuracy can be improved.

  Although the application to the rear image 74 captured by the rear imaging unit 16 has been described in the above embodiment, the front image 70 captured by the front imaging unit 12, the left side image 72 captured by the left side imaging unit 14, and the right side Application to the right side image 76 captured by the imaging unit 18 is also possible. When applying this invention to the back image 74 which the back imaging part 16 imaged, it is useful at the time of the backward parking in a parking lot, for example. When the present invention is applied to the forward image 70 captured by the forward imaging unit 12, it is useful, for example, in forward parking in a parking lot. When the present invention is applied to the left-side image 72 captured by the left-side imaging unit 14 or the right-side image 76 captured by the right-side imaging unit 18, it is useful, for example, in sideways parking or passing on narrow roads. It is.

  The present invention has been described above based on the embodiments. It is understood by those skilled in the art that this embodiment is an exemplification, and that various modifications can be made to the combination of each component and each processing process, and such a modification is also within the scope of the present invention. .

  50 Vehicle Display Device, 52 Image Acquisition Unit, 54 Distance Acquisition Unit, 56 Image Processing Unit, 58 Viewpoint Conversion Processing Unit, 60 Display Processing Unit, 62 Display Unit, 70 Forward Image, 72 Left Direction Image, 74 Rear Image, 76 Right side image, 78 俯瞰 image, 100 vehicles.

Claims (6)

An image acquisition unit for acquiring an image obtained by imaging the surroundings of the vehicle;
A distance acquisition unit that acquires a distance to an object captured in an image acquired by the image acquisition unit;
An image processing unit configured to delete a pattern included in the image acquired by the image acquisition unit and located below the object from the image according to the distance acquired by the distance acquisition unit;
A viewpoint conversion processing unit that converts a viewpoint as viewed from above the vehicle with respect to the image whose pattern has been deleted by the image processing unit;
A display processing unit that causes the display unit to display an image converted by the viewpoint conversion processing unit;
A display device for a vehicle, comprising:   The display device for a vehicle according to claim 1, wherein the image processing unit erases the pattern from the image when the distance acquired by the distance acquiring unit is shorter than a threshold.   The image processing unit derives the distance between the object and the pattern based on the distance acquired by the distance acquisition unit, and deletes the pattern from the image when the derived distance is shorter than a threshold. A display device for a vehicle according to claim 1, characterized in that.   The said image processing part sets a threshold value according to the height from the ground of the imaging device which imaged the image acquired in the said image acquisition part, The vehicle of Claim 2 or 3 characterized by the above-mentioned. Display device. The object for which the distance acquisition unit acquires a distance is another vehicle.
The vehicle display device according to any one of claims 1 to 4, wherein the pattern to be erased in the image processing unit is a frame line drawn on the ground. Acquiring an image obtained by imaging the surroundings of the vehicle;
Acquiring a distance to an object captured in the acquired image;
Erasing from the image a pattern that is included in the acquired image and is located below the object, according to the acquired distance;
Transforming the viewpoint as seen from above the vehicle with respect to the image from which the pattern has been eliminated;
Displaying the converted image on the display unit;
A display method for a vehicle comprising:

Images