JP4847913B2 - Work machine periphery monitoring device - Google Patents

Description

  The present invention relates to a work machine periphery monitoring device that monitors obstacles around a work machine, and in particular, can easily and accurately grasp the positional relationship between a work machine and an obstacle, and instantly determines whether or not a front work machine is turned. It is related with the surrounding information presentation means which makes it possible to judge.

  In order to prevent a contact accident between a working machine such as a hydraulic excavator and surrounding workers, a monitoring camera system for the working machine has been proposed (for example, see Patent Document 1).

  In this surveillance camera system, the side camera image of the work machine and the mirror image of the rear camera image are displayed on the cab monitor. The displayed camera image is switched according to the driving operation.

  However, in the case of this type of device, the driver has to determine whether or not the driver is in a dangerous situation by looking at the camera image of the monitor in the driver's room, which may be a burden on the driver.

  In view of this, there has been proposed a monitoring device that includes obstacle detection means such as a proximity sensor in addition to the camera, and displays a camera image on the monitor in the cab 1f when the obstacle is detected to notify the driver of the danger ( For example, see Patent Document 2).

  This device is provided with a camera that images the detection area of each proximity sensor corresponding to a plurality of proximity sensors, and when the presence of an object is detected by any of the proximity sensors, an image captured by the corresponding camera is selectively used. Display and inform the driver of the danger.

Japanese Patent Laid-Open No. 2001-140286 (pages 3 to 5 and FIGS. 1 to 4) JP 2002-327470 A (pages 3 to 5 and FIGS. 1 to 4)

  However, in the apparatus of Patent Document 2 that selectively displays a camera image of an area where an object is detected, if there are a plurality of workers or structures on the work site, it is only necessary to display the image by the camera. It was difficult to determine whether a worker or a structure was detected as an obstacle, requiring the driver's attention and judgment, which could lead to a reduction in work efficiency.

  Further, in an actual work site, there are cases where structures necessary for work are installed at positions where the degree of danger is relatively low around the work machine. In this case, in the apparatus of Patent Document 2, a structure that is not particularly dangerous is also detected as an obstacle, an alarm is issued, and the camera image remains displayed.

  If the work is continued in that state, even if a worker enters there and a truly dangerous situation occurs, there will be no particular change in the alarm or the camera image display, so the worker has been overlooked. There is a possibility.

  Furthermore, when the driver looks at the camera image on the driver's cab monitor to determine whether or not the driver is in a dangerous situation, the driver can grasp the entire periphery of the work machine simply by displaying the camera image. It is difficult to confirm in which position around the front work machine is working, how large and long it is, and in which direction the lower traveling body is located.

  As a result, if there is an obstacle within the monitoring range, it is difficult to instantly grasp the positional relationship from the front work machine to the obstacle, and it takes time to determine whether or not the front work machine can turn, and the work efficiency There was a risk of lowering

  An object of the present invention is to provide a work machine periphery monitoring device that can easily and accurately grasp the positional relationship between a work machine and an obstacle and can instantaneously determine whether or not a front work machine is turned. .

In order to achieve the above object, the present invention measures a work machine posture capturing means for capturing the posture and shape of a work machine at the time of work, a camera for photographing the periphery of the work machine, an obstacle around the work machine and its position. an obstacle detecting unit for a work machine attitude posture of the work machine based on information from the acquisition means, shape and image drawing the working range, the work machine around taken by the camera image, the obstacle detection A work machine periphery monitoring device is proposed that includes a display image generating unit that generates a display image obtained by synthesizing an image of an obstacle detected by the unit and a display unit that displays the generated image.

In particular, wherein the image position correction means for correcting the rotation of the image captured by the camera on the basis of information from the working machine position capturing means, provided between the camera and the display image generating means.

  A panorama synthesizing unit that integrates images captured by the camera into a panoramic image is provided in front of the display image generating unit.

  When a camera that does not reach the entire circumference of the area that can be photographed at one time is arranged in the blind spot direction of the driver, the camera is equipped with an image storage means for storing an image photographed by the camera. Are combined with the current image stored in the image storage means.

  When a camera whose range that can be shot at one time does not reach the entire circumference is arranged in the driver's blind spot direction, it is equipped with image storage means for storing images taken by the camera, and the panorama synthesis means takes pictures with the camera. A past image stored in the image storage means and a current image in which the camera is photographing the blind spot direction may be combined.

An overhead conversion means for converting an image taken by the camera into an overhead image from an upper viewpoint of the work machine can be provided in front of the display image generation means . In addition, when combining an image captured by the imaging unit with the past captured by the imaging unit, it is possible to generate an image by changing the display image over time.

  It is also possible to provide a panorama composition unit that integrates images captured by the camera into a panorama image between the overhead conversion unit and the display image generation unit.

  When a camera that does not reach the entire circumference of the area that can be photographed at one time is arranged in the blind spot direction of the driver, the camera is equipped with an image storage means for storing an image photographed by the camera. Are combined with the current image stored in the image storage means.

  When a camera whose range that can be shot at one time does not reach the entire circumference is arranged in the driver's blind spot direction, it is equipped with image storage means for storing images taken by the camera, and the panorama synthesis means takes pictures with the camera. A past image stored in the image storage means and a current image in which the camera is photographing the blind spot direction may be combined.

  The present invention also provides a work machine posture capturing means for capturing the posture and shape of the work machine at the time of operation, a camera for photographing the periphery of the work machine, and an obstacle around the work machine from an image of the periphery of the work machine. And an obstacle detection means for extracting the position thereof, an image for drawing the posture and shape of the work machine and the work range based on information from the work machine posture taking means, an image of the periphery of the work machine taken by the camera, A work machine periphery monitoring device is proposed that includes a display image generation unit that generates a display image by synthesizing an image that draws an obstacle extracted by the obstacle detection unit, and a display unit that displays the generated image.

  According to the present invention, the presence / absence and position of an obstacle in the monitoring range around the work machine, the work position / size / length of the front work machine, the position of the lower traveling body, etc. are presented to the driver. Therefore, the positional relationship between the work machine and the obstacle can be grasped at a glance, and the propriety of turning of the front work machine can be determined accurately and instantaneously, thereby improving the work efficiency and safety.

  Next, with reference to FIGS. 1-24, the Example of the working machine periphery monitoring apparatus by this invention is described.

  FIG. 1 is a perspective view showing a hydraulic excavator as an example of a working machine to which the present invention is applied.

  The hydraulic excavator includes an articulated front work machine 1A including a boom 1a, an arm 1b, and a bucket 1c that rotate in a vertical direction, and a vehicle body 1B including an upper swing body 1d and a lower traveling body 1e. An cab 1f is mounted on the upper swing body 1d. The base of the boom 1a of the front work machine 1A is supported by the front part of the upper swing body 1d.

  The boom 1a, the arm 1b, the bucket 1c, the upper swing body 1d, and the lower travel body 1e are respectively provided by actuators such as hydraulic actuators 3a, 3b, 3c, a swing motor 3d not shown here, and left and right travel motors 3e, 3f. Driven. The boom 1a, the arm 1b, the bucket 1c, and the upper swing body 1d are provided with angle detectors 8a, 8b, 8c, and 8d that detect respective rotation angles.

  FIG. 2 is a plan view of the excavator of FIG.

  A right side monitoring camera 13a and a rear side monitoring camera 13b of the vehicle body 1B are provided on the upper part of the upper swing body 1d, and images the monitoring ranges 12a and 12b. A dotted line 14 indicates a work range of the front work machine 1A. When the worker enters the work range of the front work machine 1A, the front work machine 1A and the worker may collide.

  FIG. 3 is a plan view showing a state in which the upper swing body 1d of the hydraulic excavator in FIG. 1 is turned 90 ° to the right.

  Of the vehicle body 1B, the upper turning body 1d turns, and the lower traveling body 1e maintains the same posture. As described above, when the upper swing body 1d is swung from the state shown in FIG. 2 to the state shown in FIG. 3, the video imaged in FIG. 2 becomes past information, and the video imaged in FIG.

  FIG. 4 is a perspective view showing an example of a work site of a hydraulic excavator.

  The hydraulic excavator is provided with a camera 13a that captures the right side, which is a blind spot from the driver, and a camera 13b that captures the rear, and monitors the operating range of the front working machine 1A of the hydraulic excavator and its surroundings. The front work machine 1A rotates following the turning of the upper turning body 1d. Further, an obstacle 40 such as a person and an obstacle 41 exist inside the fences 91 to 94.

The camera that monitors the periphery of the vehicle body 1B may be only the camera 13b that captures the rear if the range that can be captured is wide. If the installation cost is sufficient, in addition to the cameras 13a and 13b, a camera 13c for photographing the left side can be installed. If four cameras 13a to 13d are installed on the right side, rear side, and left side front of the excavator, the entire periphery of the current work site can be photographed at a time.

  FIG. 5 is a plan view showing an overhead image of the work site of the excavator of FIG.

  A right side monitoring camera 13a and a rear side monitoring camera 13b of the vehicle body 1B are provided on the upper part of the upper swing body 1d, and images the monitoring ranges 12a and 12b.

  FIG. 6 is a block diagram showing a system configuration of Embodiment 1 of the work machine surrounding monitoring apparatus according to the present invention.

  The work machine surroundings monitoring apparatus according to the present embodiment includes an image input unit 100, a distortion correction unit 200, a work machine posture capturing unit 300, an image position correction unit 400, an overhead conversion unit 500, a panorama synthesis unit 600, An obstacle detection unit 700, a display image generation unit 800, and a display device 900 are included.

  The image input means 100 stores a video signal obtained by photographing the target object 40 of the monitoring target scene 20 by the camera 13a, and stores a video signal 32 obtained by photographing the target object 41 of the monitoring target scene 21 by the camera 13b.

  The distortion correction unit 200 corrects the distortion of the stored image when distortion occurs around the image. If there is no distortion, the processing procedure of the distortion correction means 200 may be omitted. The distortion correction means 200 can be realized by using, for example, the technology disclosed in Japanese Patent Application Laid-Open No. 2006-59270.

  The work machine posture taking-in means 300 takes in data from angle detectors 8a, 8b, 8c, and 8d that detect the rotation angles of the boom 1a, arm 1b, and bucket 1c, and the size and length of the front work machine 1A. Is calculated from the angle detector 8d that detects the rotation angle of the upper swing body 1d, and the size and length of the front work machine 1A and the swing direction of the upper swing body 1d are determined. .

  The image position correction means 400 corrects the inclination and rotation of the images of the cameras 13a and 13b in which the work machine is operating because they are affected by the inclination and rotation according to the attitude data of the upper swing body 1d. Create the finished image.

  The overhead conversion unit 500 converts the image corrected by the image position correction unit 400 into an overhead image. The overhead conversion unit 500 can be realized by using, for example, the technique of Japanese Patent Application Laid-Open No. 2006-48451.

  The panorama composition unit 600 creates a panorama composite image obtained by integrating the overhead view images.

  The obstacle detection means 700 detects the obstacle 40 or the obstacle 41 as an obstacle. The obstacle detection means 700 can be realized by using, for example, the technique of Japanese Patent Laid-Open No. 6-333049.

  The display image generation unit 800 converts the position and distance of the obstacle detected by the obstacle detection unit 700 into the panoramic image converted by the panorama synthesis unit 600 and the size of the front work machine 1A calculated by the work machine posture capturing unit 300. The vehicle body 1B reflecting the length and length and the turning direction of the upper turning body 1d from the lower traveling body 1e is superimposed on the center of the panoramic image to generate a display image and output it to the display device 900.

  The display device 900 calculates the size and length of the front work machine 1A using data such as the angle detector 8a, reflects the calculated value of the front work machine 1A with the vehicle body 1B at the center, A panoramic image superimposed on the appearance position and distance of the obstacle 41 is displayed and presented to the operator 1000 in the cab 1f.

  The driver 1000 during work can clearly grasp the positional relationship between the machine and the obstacle at a glance when there is an obstacle, and can accurately determine the propriety of the turning of the front work machine 1A instantly. It is possible to avoid and increase work efficiency.

  FIG. 7 is a flowchart illustrating a processing procedure in the work machine posture capturing unit 300 according to the first embodiment.

  In step 301, the work machine posture taking-in means 300 takes in the output θ1 of the angle detector 8a that detects the rotation angle of the boom 1a. In step 302, the output of the angle detector 8b that detects the rotation angle of the arm 1b. In step 303, the output θ3 of the angle detector 8c that detects the rotation angle of the bucket 1c is acquired. In step 304, the output θ4 of the angle detector 8d that detects the rotation angle of the upper swing body 1d is acquired. take in.

  In step 305, the work machine posture taking-in means 300 calculates the tip coordinates of the bucket 1c from θ1, θ2, θ3, and θ4. In step 306, the size and length of the front work machine 1A are calculated using θ1, θ2, and θ3. The turn direction of the upper swing body 1d from the lower traveling body 1e is calculated using θ4.

  As a result, the work machine posture capturing means 300 calculates the work range 14 of the front work machine 1A in the vehicle body 1B and the relative relationship between the turning direction of the upper swing body 1d and the lower traveling body 1e, and the turning state of the work machine And determine the turning range.

  FIG. 8 is a flowchart illustrating a processing procedure in the image position correcting unit 400 according to the first embodiment.

  The image position correcting means 400 calculates the tilt / rotation from the current working machine posture of the camera 13a in step 401 based on the posture data of the upper swing body 1d taken from the working machine posture taking means 300, and in step 402. The tilt / rotation is calculated from the current working machine posture of the camera 13b, and in step 403, the tilt / rotation is calculated from the past working machine posture of the camera 13a. In step 404, the previous working machine posture of the camera 13a is calculated. Calculate tilt and rotation.

  Next, using the image created by the distortion correction means 200, a current right side position corrected image of the camera 13a is created in step 405, and a current right side position corrected image of the camera 13a is created in step 406. In step 407, a past right-side position corrected image of the camera 13a is created. In step 408, a past rear position-corrected image of the camera 13b is created.

  FIG. 9 is a view showing an image obtained by photographing the entire periphery of the work site of the excavator of FIG. 4 with a camera.

  When a work machine such as a hydraulic excavator is moved to the site, the upper swing body 1d and the lower travel body 1e of the vehicle body 1B travel in the same direction. When the work position is reached, the lower traveling body 1e is fixed and the upper turning body 1d is turned to perform the intended work.

  If the user turns around the work site and makes a round, the camera 13b installed behind the upper swing body 1d can take an image of the work site. When only the camera 13b is installed behind the revolving structure 1d, the image behind the revolving structure 1d is the current image, and the other peripheral images are past images taken by the camera 13b.

  When the hydraulic excavator is working at a certain time, the right side monitoring camera 13a captures the right side scene 22, and the rear monitoring camera 13b captures the rear scene 23. Next, when the excavator moves 30 after a predetermined time has elapsed, the right side monitoring camera 13a captures the right side scene 20 and the rear monitoring camera 13b captures the rear scene 21.

  When the obstacle 40 exists in the scene 20 and the obstacle 41 exists in the scene 21, when the obstacle detection unit 700 detects the obstacles 40 and 41, the shooting scene 20 and the scene 21 are the current input images. .

  Therefore, the entire scene around the work site of the hydraulic excavator at the time when the obstacle is detected can be created using the current right side scene 20 and the rear scene 21, and the past right side scene 22 and the rear scene 23.

  If the camera 13 is installed on the front, rear, left side, and right side of the excavator, the entire periphery of the current work site is photographed. On the other hand, if there is only one camera 13 behind the hydraulic excavator, the entire surroundings of the work site can be created for the front scene, left side scene, and right side scene using past images. Therefore, the entire periphery of the work machine can be monitored.

  Usually, a work machine such as a hydraulic excavator has a good view of the monitoring scene in the front and left visual fields from the operator operating in the cab 1f and can be seen visually.

  In this embodiment, the cameras 13a and 13b are installed on the right side and the rear side in the blind spot from the operator, the right side and the rear side are monitored by the camera 13, and the current shooting scenes on the right side and the rear side are displayed.

  In this embodiment, the cameras 13a and 13b are installed on the right side and the rear side in the blind spot from the operator, the right side and the rear side are monitored by the camera 13, the right side and the rear side display the current shooting scene, and the front side For the monitoring scene on the left side, past images may be used.

  If the operator confirms the right and rear scenes with the current image on the display device 900 and visually confirms the front and left scenes, the entire monitoring periphery can be confirmed in real time.

  FIG. 10 is a diagram illustrating a relationship between an image captured by the right-side monitoring camera and an overhead image.

  Assuming that the right side monitoring camera 13a is moved right above the scene 20, the scene 20 becomes an overhead image 20a, and the appearance position of the obstacle 40 in the monitoring scene becomes clear.

  FIG. 11 is a diagram illustrating a relationship between an image captured by the rear monitoring camera and an overhead image.

  Assuming that the rear monitoring camera 13b is moved directly above the scene 21, the scene 21 becomes an overhead image 21a, and the appearance position of the obstacle 41 in the monitoring scene becomes clear.

  FIG. 12 is a flowchart illustrating a processing procedure in the panorama composition unit 600 according to the first embodiment.

  The panorama synthesizing unit 600 reads the current right side bird's-eye view image of the camera 13a at step 601 and reads the current rear side bird's-eye image of the camera 13b at step 602. A similar pattern is searched and its position is calculated. In step 604, the current panoramic image is generated by superimposing the videos at the position.

  The panorama synthesizing unit 600 reads the past right side bird's-eye view image of the camera 13a in step 605, reads the past rear side bird's-eye image of the camera 13b in step 606, and in step 607, the same or the same between the read images. A similar pattern is searched and its position is calculated. In step 608, the past panoramic images are generated by superimposing the video images at the positions.

  In step 609, the panorama synthesizing unit 600 searches for the same or similar pattern between the current panorama image generated in step 604 and the past panorama image generated in step 608, and calculates its position. In step 610, from the position information calculated in step 609, the current panorama image and the past panorama image are overlapped to generate a display panorama image.

  FIG. 13 is a diagram for explaining a panoramic image synthesizing procedure in the panoramic synthesizing means 600.

  In the right side bird's-eye view image 20 of the camera 13a and the rear bird's-eye view image 21 of the camera 13b, the pattern 520 of the image 20 and the pattern 521 of the image 21 are the same or similar patterns. Can be generated.

  Next, since the pattern 522 of the image 22 and the pattern 523 of the image 23 are the same or similar in the past right side bird's-eye view image 22 and the past rear bird's-eye view image 23, when these are overlapped, the past panoramic image 531 is superimposed. Can be generated.

  Finally, in the current panorama image 530 and the past panorama image 531, the pattern 524 of the panorama image 530 and the pattern 525 of the panorama image 531 are the same or similar patterns. Can be generated.

  In order to prevent matching errors, it may be used that the pattern 526 of the panorama image 530 and the pattern 527 of the panorama image 531 are the same or similar patterns.

  However, if the same or similar pattern does not exist, the images cannot be overlapped, and an image interpolated with the past video is generated for that portion. Alternatively, for example, it may be clearly indicated that the image of the portion does not exist by leaving it in black or white.

  FIG. 14 is a perspective view showing an example in which an obstacle detector is externally attached in place of the obstacle detection means 700 for detecting an obstacle from a panoramic image.

  Instead of the obstacle detection means 700 for detecting an obstacle from a panoramic image, obstacle detectors 10a and 10b are installed on the right and rear surfaces of the upper swing body 1d, and obstacle detection is performed on the left side surface of the upper swing body 1d. A vessel 10c may be installed.

  The obstacle detectors 10a, 10b, and 10c are proximity sensors that detect obstacles such as surrounding workers and structures and obtain the position of the obstacle from the distance and direction, and are, for example, laser radars.

  Further, only one obstacle detector 10b may be used. In that case, the rear monitoring camera 13b of the upper swing body 1d is also installed.

  In the above embodiment, the obstacle detection means 700 detects an obstacle from the image of the camera by the image processing device 50. The present invention is not limited to a method using a camera image. The means for detecting obstacles around the vehicle body 1B can be used as long as it is a proximity sensor such as a millimeter wave radar, an ultrasonic sensor, or an infrared sensor.

  In the said Example, the example provided with the obstacle detector 10 and the camera 13 1 each in the upper revolving body 1d side and back was shown. The position and number of the obstacle detector 10 and the camera 13 are not limited to this embodiment. The obstacle detector 10 and the camera 13 may be provided only on the side or only on the rear side, or may be mounted at more locations and monitored over a wider area. In that case, the camera image area may be increased in the display device 900, or a plurality of display devices 900 may be provided.

  FIG. 15 is a flowchart illustrating a processing procedure in the display image generation unit 800 according to the first embodiment.

  The display image generation means 800 calculates the work range 14 from the size and length of the front work machine 1A in step 801, and uses the length of the front work machine 1A and the direction from the lower traveling body 1e in step 802. Then, with the front work machine 1A in front, the positional relationship with the lower traveling body 1e is determined, and in step 803, an overhead view image that is drawing data of the vehicle body 1B is created.

  In step 804, the display image generation unit 800 calculates the positions of the obstacles 40 and 41 from the distance data between the cab 1f and the obstacles 40 and 41 using the detection data of the obstacle detection unit 700, To draw.

  In step 805, a display image is generated by superimposing the image in which the front work machine 1 </ b> A of the vehicle body 1 </ b> B is placed at the center, the monitoring ranges 12 a and 12 b, and the obstacles 40 and 41 on the display device 900. Output.

  According to the bird's-eye view image generated in this way and displayed on the display device 900, the operator 1000 in the operator cab 1f is behind or on the right side of the obstacles 40 and 41 while looking at the front at the center position. Can be recognized at a glance, and whether or not the work machine can be turned can be determined.

  In addition, when the obstacle detection part 700 detects the obstacles 40 and 41, while displaying on the display apparatus 900, you may alert | report to an operator with an audio | voice. Further, at the start of turning, an alarm such as the presence of an obstacle may be output, or the movement of the obstacle may be output by voice.

  FIG. 16 is a screen example when an obstacle is detected in the screen example of the work machine periphery monitoring device displayed on the display device 900.

  When the cameras 13a and 13 capture the target objects 40 and 41 of the monitoring target scenes 20 and 21, the overhead conversion unit 500 converts the captured video signal into an overhead view, and the panorama synthesis unit 600 synthesizes the panorama image, and the failure is detected. The object detection means 700 detects the obstacle 40 or 41 and calculates the distance from the camera 13a, 13b to the obstacle 40, 41.

  The display image generation means 800 clearly shows that the obstacle 40 existing in the work range 14 of the front work machine 1A is involved, so that the obstacle 41 exists outside the work range 14. Display with caution.

  The left and rear images from the monitoring cameras 13a and 13b are past images taken when the upper-part turning body 1d turns. Therefore, the front image 910 and the left image 911 are enclosed in a frame so as to indicate that they are past images, and the image before j seconds, the image before i seconds, and the like are clearly shown in the frame.

  The front work machine 1A and the vehicle body 1B are displayed in a bird's-eye view image viewed from directly above in the center of the screen. The upper swing body 1d and the front work machine 1A are displayed in the upward direction regardless of the direction of the lower traveling body 1e.

  That is, the vehicle body 1B on which the operator is working is arranged in the center, and a panoramic synthesis so as to surround the periphery of the vehicle body 1B, the work range 14 of the front work machine 1A, the distance 40a to the obstacles 40, 41 , 41a are superimposed and displayed. The display device 900 may be provided with means for displaying these and generating an alarm sound.

  If nothing is displayed, it will not be possible to determine whether the perimeter monitoring device has successfully detected that there are no obstacles or whether the perimeter monitoring device has failed despite the presence of obstacles, so no obstacles will be detected. In such a case, the message “no obstacle” may be positively displayed.

  When an obstacle is detected, the functions of reproduction 902, stop 903, frame advance 904, return 905, enlargement 906, and standard 907 can be provided. Playback 902 is a function for playing back video, stop 903 is a function for stopping playback of video, frame advance 904 is a function for playing back every frame, return 905 is a function for returning backward, and enlargement 906 is for zooming in on an obstacle The display function, standard 907, is a function for returning the enlarged display portion to the standard display.

  By making full use of these functions, the driver 1000 during work grasps at a glance whether there is an obstacle and the positional relationship between the work machine and the obstacle and accurately and instantaneously determines whether the front work machine 1A is turning properly. As a result, obstacles can be reliably avoided and work efficiency and safety can be greatly improved. Further, if the enlargement function 906 is used, the details of the obstacle can be confirmed.

  FIG. 17 is a diagram illustrating a screen example of the worker periphery monitoring device displayed on the display device 900.

  The rear image and the right image are the current images. Therefore, the video 920 captured by the right camera 13a is displayed, and the video 921 captured by the rear camera 13b is displayed. Due to the angle of view of the right camera 13a and the rear camera 13b, a blind spot 912 that cannot be captured by the camera may occur. In that case, a panoramic image is generated by interpolating the area of the blind spot 912 with black or the like clearly indicating the blind spot, and images on the right side and the rear side of the driver are displayed.

  In addition, since the past video is not displayed, a blank 922 may be displayed so that it is understood that there is no video display.

  FIG. 18 is a diagram illustrating a screen example of the work machine periphery monitoring device displayed on the display device 900.

  When the obstacle 40 is detected, the front image and the left image are past images. Therefore, the display image is changed in accordance with the passage of time in the past, and the image 911b before n seconds and the image 910b before m seconds are displayed darkly. Here, since the video 911b before n seconds has passed the time from the video 910b before m seconds, it is displayed darkly. A display that can be easily grasped by the operator may be displayed, for example, it may be darkly displayed as reliability decreases with time.

  Due to the angle of view of the right camera 13a and the rear camera 13b, a blind spot 912 that cannot be captured by the camera may occur. In that case, the area of the blind spot 912 is interpolated with the video captured in the past, a panoramic image without a blind spot is generated, and the entire peripheral video of the driver is displayed.

  FIG. 19 is a diagram illustrating another screen of the work machine periphery monitoring device displayed on the display device 900.

  At the start of the work, the front image and the left image are past images, so that the start inspection item 920 is clearly indicated and the operator is made to confirm the inspection item without displaying the image in this portion.

  FIG. 20 is a diagram showing another screen of the work machine periphery monitoring device displayed on the display device 900.

  The obstacle detector 10c detects the obstacle 42 in the monitoring ranges 12a and 12b and measures the distance. In this case, since the left side monitoring camera is not provided, the obstacle 42 is superimposed and displayed on the past video 911c. With this method, the number of surveillance cameras can be reduced.

  FIG. 21 is a diagram showing still another screen example of the work machine periphery monitoring device displayed on the display device 900.

  Since the front image and the left image are past images, the image is not displayed in the portions 910d and 911d, and the hydraulic pump state 930 that is the state of the vehicle body 1B is displayed, and the vehicle body 1B is displayed to the driver. Get to know the condition. At this time, the normal upper limit 940 of the vehicle body 1B state may also be clearly indicated.

  FIG. 22 is a block diagram showing a system configuration of embodiment 2 of the work machine surrounding monitoring apparatus according to the present invention.

  In the second embodiment, the direction sensor 8e in which the direction sensor 8e is installed on the upper swing body 1d of the vehicle body 1B is used when the angle detector 8d for detecting the rotation angle of the upper swing body 1d is not provided. It is a sensor, and detects the difference of the rotation angle in that direction between the upper swing body 1d and the lower traveling body 1e.

  The working machine direction taking-in means and the front work machine length direction determining means 350 are the direction sensor 8e taking in the direction data for detecting the direction of the upper swing body 1d, and the difference in rotational angle between the lower traveling body 1e and the direction. Is detected, and the longest size and length of the front work machine 1A are determined. The lower traveling body 1e may always face fixed (forward).

  The display image generation means 800 includes the position of the obstacle detected by the obstacle detection means 700, the capturing means in the work machine direction, and the length direction determination means 350 of the front work machine in the panoramic image converted by the panorama synthesis means 600. The size and length of the front work machine 1A determined by the vehicle body 1B reflecting the turning direction of the upper turning body 1d from the lower traveling body 1e are superimposed on the center of the panoramic image to generate a display image. It is displayed on the device 900 and presented to the operator 1000 in the cab 1f.

  The driver during the operation can grasp the positional relationship between the machine and the obstacle at a glance when there is an obstacle, and can judge the propriety of turning of the front work machine 1A accurately and instantaneously, thus avoiding the obstacle with certainty. It becomes possible to increase work efficiency.

  FIG. 23 is a flowchart illustrating a processing procedure in the working machine direction taking-in means and the front work machine length direction determining means 350 according to the second embodiment.

  The work machine direction take-in means and the front work machine length direction determination means 350 include a direction data take-in procedure for detecting the direction of the upper swing body 1d and a procedure for determining the size and length of the front work machine 1A. Yes.

  In step 751, the data θ5 of the direction sensor 8e that detects the difference between the rotation angles of the upper swing body 1d and the lower traveling body 1e is captured. In step 752, the direction of the lower traveling body 1e is fixed (forward direction), and the relative angle between the upper swing body 1d and the lower traveling body 1e is calculated from θ5. In step 753, the length of the front work machine 1A is determined as the longest fixed value.

  Based on the results of steps 752 and 753, in step 704, the size and length of the front work machine 1A and the turning direction of the upper swing body 1d from the lower traveling body 1e are calculated.

  Therefore, the working range of the front working machine 1A in the vehicle body 1B and the relative relationship between the turning directions of the upper swing body 1d and the lower traveling body 1e can be calculated, and the turning state and the turning range of the work machine can be determined.

  FIG. 24 is a perspective view showing an appearance of a working machine surrounding monitoring apparatus according to Embodiment 3 of the present invention.

  In the third embodiment, more obstacle detectors are installed than cameras. That is, the obstacle detector 10b is provided on the upper part of the upper swing body 1d corresponding to the rear monitoring camera 13b of the vehicle body 1B. Further, an obstacle detector 10c is provided on the left side of the vehicle body 1B. The obstacle detectors 10b and 10c are proximity sensors such as a millimeter wave radar, an ultrasonic sensor, and an infrared sensor.

  In the said Example, the example provided with the obstacle detection means 700 and the camera 1 each in the side and back of the upper revolving body 1d was shown. The camera and obstacle detection means 700 may be provided only on the side or only on the rear side, or may be mounted at more locations and monitored over a wider area. In that case, the camera image area may be increased in one display device 900, or a plurality of display devices 900 may be provided.

  In the above embodiment, the camera image is displayed only when the obstacle detection unit 700 detects an obstacle. In a small road construction site, workers and other machines may always be engaged in some work in the vicinity of the work machine. In this case, in the configuration in which the display / non-display of the camera image is switched by the obstacle detection by the obstacle detection unit 700, the screen changes frequently, causing the driver to feel extra stress, which is counterproductive in terms of safety. There is also a risk. In such a field, the camera image may be displayed constantly regardless of whether the obstacle detection unit 700 detects the obstacle.

1 is a perspective view showing a hydraulic excavator as an example of a work machine to which the present invention is applied. It is a top view of the hydraulic excavator of FIG. FIG. 2 is a plan view showing a state in which an upper swing body 1d of the hydraulic excavator in FIG. 1 is turned 90 ° to the right. It is a perspective view which shows an example of the working site of a hydraulic excavator. It is a top view which shows the bird's-eye view image of the working site of the hydraulic excavator of FIG. It is a block diagram which shows the system | strain structure of Example 1 of the working machine surroundings monitoring apparatus by this invention. 6 is a flowchart illustrating a processing procedure in the work machine posture capturing unit 300 according to the first embodiment. 6 is a flowchart illustrating a processing procedure in the image position correcting unit 400 according to the first exemplary embodiment. It is a figure which shows the image which image | photographed the whole circumference | surroundings of the work site of the hydraulic excavator of FIG. It is a figure which shows the relationship between the image image | photographed with the right side monitoring camera, and a bird's-eye view image. It is a figure which shows the relationship between the image image | photographed with the back monitoring camera, and a bird's-eye view image. 6 is a flowchart illustrating a processing procedure in the panorama composition unit 600 according to the first embodiment. It is a figure explaining the panorama image synthetic | combination procedure in a panorama synthetic | combination means. It is a perspective view which shows the example which replaces with the said obstacle detection means 700 which detects an obstacle from a panoramic image, and attaches an obstacle detector externally. 6 is a flowchart illustrating a processing procedure in a display image generation unit 800 according to the first embodiment. It is an example of a screen when an obstacle is detected in the screen example of the work machine periphery monitoring device displayed on the display device 900. 6 is a diagram showing an example of a screen of a work machine periphery monitoring device displayed on a display device 900. FIG. 6 is a diagram showing an example of a screen of a work machine periphery monitoring device displayed on a display device 900. FIG. It is an example of a screen when an obstacle is detected in the screen example of the work machine periphery monitoring device displayed on the display device 900. 6 is a diagram showing another screen of the work machine periphery monitoring device displayed on the display device 900. FIG. 12 is a diagram showing still another screen example of the work machine periphery monitoring device displayed on the display device 900. FIG. It is a block diagram which shows the system | strain structure of Example 2 of the working machine surroundings monitoring apparatus by this invention. It is a flowchart which shows the process sequence in the working machine direction taking-in means of Example 2, and the length direction determination means 350 of a front work machine. It is a perspective view which shows the external appearance of Example 3 of the working machine surroundings monitoring apparatus by this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1A Front work machine 1B Car body 1a Boom 1b Arm 1c Bucket 1d Upper turning body 1e Lower traveling body 1f Driver's cab 3a, 3b, 3c Hydraulic actuator 3e, 3f Traveling motors 8a, 8b, 8c, 8d Angle detector 8e Direction sensor 10a, 10b, 10c Obstacle detector 12a, 12b Camera monitoring range 13a, 13b Camera 14 Work range 20 Current right side scene 21 Current rear scene 22 Past right side scene 23 Past rear scenes 40, 41 Worker or obstacle Object 50 Image processing devices 91-94 Fence 100 Image input means 200 Distortion correction means 300 Work machine posture take-in means 350 Work machine direction take-in means and front work machine length direction determination means 400 Image position correction means 500 Overhead conversion means 600 Panorama composition means 700 Obstacle detection means 800 Display image generating means 900 Display device

Claims (15)

Work machine posture taking means for taking in the posture and shape of the work machine at the time of work,
A camera for photographing the periphery of the work machine;
Obstacle detection means for measuring an obstacle around the work machine and its position;
Based on the information from the work machine posture taking-in means, an image for drawing the posture, shape and work range of the work machine, an image around the work machine taken by the camera, and detected by the obstacle detection means Display image generating means for generating a display image obtained by synthesizing the image for drawing the obstruction;
Display means for displaying the generated image ;
And an image position correction means for correcting the rotation of the image captured by the camera based on information from the working machine position capturing means,
A work machine periphery monitoring device, wherein the image position correcting means is provided between the camera and the display image generating means. In the work machine periphery monitoring device according to claim 1,
A work machine periphery monitoring device, wherein the image position correcting means is provided between the camera and the obstacle detecting means. In the work machine periphery monitoring device according to claim 1 ,
A work machine periphery monitoring apparatus , comprising: panorama synthesis means for integrating panoramic images obtained by integrating images around the work machine photographed by the camera before the display image generation means. In the work machine periphery monitoring device according to claim 3,
Image storage means for storing an image around the work machine taken by the camera;
Before Symbol camera is disposed in the blind spot direction of the driver,
The panorama means, work machine environment monitoring apparatus characterized by combining the current image the before and Symbol past image stored in the image storage means the camera is shooting. In the work machine periphery monitoring device according to claim 1 ,
Work machine environment monitoring, characterized in that the overhead conversion means for converting the image of the working machine around taken by the camera on the overhead view image from the upper viewpoint of the working machine, is provided in front of said display image generating means apparatus. In the work machine periphery monitoring device according to claim 5,
A panoramic image synthesizing unit that integrates images around the work machine photographed by the camera into a panoramic image;
  A work machine periphery monitoring device, wherein the panoramic image synthesis means is provided between the overhead view conversion means and the obstacle detection means. In the work machine periphery monitoring device according to claim 1 ,
If the past image and the camera taken by the camera combines the current image being captured, the work machine around depending on the time, and generates an image by changing the display image Monitoring device. In the work machine periphery monitoring device according to claim 3 ,
Having an overhead conversion means for converting an image around the work machine imaged by the camera into an overhead image from an upper viewpoint of the work machine;
The pre-Symbol panorama means, work machine environment monitoring apparatus is characterized in that provided between the display image generating means and the overview transformation means. Work machine posture taking means for taking in the posture and shape of the work machine at the time of work,
A camera for photographing the periphery of the work machine;
Obstacle detection means for extracting obstacles around the work machine and their positions from an image around the work machine photographed by the camera;
Based on information from the work machine posture capturing means, an image that draws the posture, shape, and work range of the work machine, an image around the work machine taken by the camera, and the obstacle detection means are extracted. Display image generating means for generating a display image obtained by synthesizing the image for drawing the obstruction;
Display means for displaying the generated image;
Image position correction means for correcting rotation of an image around the work machine photographed by the camera based on information from the work machine posture capturing means ;
Wherein the image position correction means, work machine environment monitoring device being characterized in that disposed between the camera and the front Symbol display image generating unit. In the work machine periphery monitoring device according to claim 9,
A work machine periphery monitoring device, wherein the image position correcting means is provided between the camera and the obstacle detecting means. In the work machine periphery monitoring device according to claim 9 ,
A work machine periphery monitoring apparatus , comprising: panorama synthesis means for integrating panoramic images obtained by integrating images around the work machine photographed by the camera before the display image generation means. In the work machine periphery monitoring device according to claim 11 ,
Image storage means for storing an image around the work machine taken by the camera;
Before Symbol camera is disposed in the blind spot direction of the driver,
The panorama means, work machine environment monitoring apparatus characterized by combining the current image the before and Symbol past image stored in the image storage means the camera is shooting. In the work machine periphery monitoring device according to claim 9 ,
Work machine environment monitoring, characterized in that the overhead conversion means for converting the image of the working machine around taken by the camera on the overhead view image from the upper viewpoint of the working machine, is provided in front of said display image generating means apparatus. In the work machine periphery monitoring device according to claim 13,
Panoramic composition means that integrates images around the work machine taken by the camera into a panoramic image;
It said panorama means, work machine environment monitoring apparatus is characterized in that provided between the front Symbol overview transformation means the obstacle detecting means. In the work machine periphery monitoring device according to claim 11 ,
Having an overhead conversion means for converting an image around the work machine imaged by the camera into an overhead image from an upper viewpoint of the work machine;
A work machine periphery monitoring device , wherein the panorama composition means is provided between the overhead view conversion means and the display image generation means .

Images