JP6479344B2 - Object detection system for construction machinery - Google Patents

Description

  The present invention relates to an object detection system for a construction machine that detects an object existing around a construction machine including a turning mechanism.

  A system is known in which when a laser scanner detects an object that has entered a warning area of a construction machine, the camera is rotated to photograph the object, and an image captured by the camera is displayed on a monitor (Patent Document). 1).

  In this system, in order to monitor a blind spot area behind the construction machine, a laser scanner is installed at the rear part of the construction machine so as to protrude from the side wall of the vehicle body.

JP-A-8-160127

  However, since the measurement range of the laser scanner attached to the vehicle body side wall is limited by the vehicle body side wall itself in the above-described system, it is necessary to install a plurality of laser scanners in order to cover the entire rear blind spot area. Not.

  In view of the above, it is desired to provide an object detection system for construction machinery that can cover a wider measurement range with a smaller number of scanning distance measuring devices.

An object detection system for a construction machine according to an embodiment of the present invention is an object detection system for a construction machine that detects an object existing around a construction machine including an upper swing body mounted on a lower traveling body via a swing mechanism. a is attached to the underside of the upper rotating body, it has a device for detecting an object.

  By the above-described means, an object detection system for construction machinery is provided that allows a wider measurement range to be covered by a smaller number of scanning distance measurement devices.

1 is a side view of an excavator on which an object detection system according to an embodiment of the present invention is mounted. It is a block diagram which shows the structural example of an object detection system. It is the schematic which shows the structural example of a scanning type distance measuring device. It is an excavator top view which shows the influence which a turning mechanism has on the measurement range of a scanning distance measuring device. It is a shovel top view which shows the influence which the obstruction on a lower traveling body has on the measurement range of a scanning type distance measuring device. It is a side view which shows the structural example of the shovel provided with the laser beam protection structure. It is a side view which shows the structural example of the shovel provided with another laser beam protection structure. It is a top view of the shovel showing the influence of the laser beam protection structure on the measurement range of the scanning distance measuring device.

  FIG. 1 is a side view of an excavator on which an object detection system 100 according to an embodiment of the present invention is mounted. An upper swing body 3 is mounted on the lower traveling body 1 of the excavator via a swing mechanism 2. A boom 4 is attached to the upper swing body 3. An arm 5 is attached to the tip of the boom 4, and a bucket 6 is attached to the tip of the arm 5. The boom 4, the arm 5, and the bucket 6 constitute a digging attachment, and are hydraulically driven by the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9, respectively. Further, the upper swing body 3 is provided with a cabin 10 and is mounted with a power source such as an engine. A scanning distance measuring device 40 is attached to the lower surface of the counterweight that constitutes the rear end portion of the upper swing body 3. In addition, imaging devices 41 are attached to three locations, the rear end upper portion, the left end upper portion, and the right end upper portion of the upper swing body 3. A controller 30 and an output device 50 are installed in the cabin 10.

  FIG. 2 is a block diagram illustrating a configuration example of the object detection system 100. The object detection system 100 mainly includes a controller 30, a scanning distance measuring device 40, an imaging device 41, and an output device 50.

  The controller 30 is a control device that performs drive control of the shovel. In the present embodiment, the controller 30 is configured by an arithmetic processing unit including a CPU and an internal memory, and realizes various functions by causing the CPU to execute a drive control program stored in the internal memory.

  Further, the controller 30 determines whether an object exists around the shovel based on the outputs of the various devices, and controls the various devices according to the determination result. In the present embodiment, a case where “person” is particularly targeted as an object will be described. Specifically, the controller 30 receives the outputs of the scanning distance measuring device 40 and the imaging device 41, and executes software programs corresponding to the object detection unit 31, the human candidate extraction unit 32, and the human identification unit 33, respectively. Run. Then, drive control of the shovel is executed according to the execution result, or various information is output from the output device 50.

  The scanning distance measuring device 40 is a device that measures the distance to an object existing around the excavator, and outputs measurement data to the controller 30. In this embodiment, the scanning distance measuring device 40 is a two-dimensional scanning laser range finder using a semiconductor laser. Specifically, the scanning distance measuring device 40 is attached to the lower surface of the counterweight so that the optical axis of the semiconductor laser passes through the gap between the lower traveling body 1 and the upper swing body 3. More specifically, the scanning distance measuring device 40 is attached to the center portion in the vehicle width direction at the rear end portion of the lower surface of the counterweight.

  FIG. 3 is a schematic diagram illustrating a configuration example of the scanning distance measuring device 40, FIG. 3 (A) is a side view thereof, and FIG. 3 (B) is a plane indicated by a one-dot chain line in FIG. 3 (A). The figure which looked at from the III-III direction is shown.

  As shown in FIG. 3, the scanning distance measuring device 40 mainly includes a main body portion 40a, an optical window portion 40b, and a lid portion 40c. The main body 40a is a part that houses a semiconductor laser generator, a mirror rotation motor, a light receiver (none of which are shown), and the like. The optical window portion 40b is a portion that houses a rotating mirror (not shown) that is driven to rotate around the Z axis by a mirror rotating motor, and the casing is made of a material that can transmit laser light. . Then, the scanning distance measuring device 40 reflects the laser beam 40X generated by the semiconductor laser generator with a rotating mirror and irradiates the laser beam 40X radially around the measurement axis parallel to the XY plane and parallel to the Z axis. To do. In the present embodiment, the measurement range of the scanning distance measuring device 40 corresponds to a circular area having a predetermined radius (for example, 30 meters) around the measurement axis on the XY plane. Note that the measurement range may be a sector region having a central angle of less than 360 degrees.

  With this configuration, the scanning distance measuring device 40 derives the irradiation direction of the laser light 40X from the rotation angle of the rotating mirror, and also the distance from the flight time of the laser light to the reflector (hereinafter referred to as “reflector distance”). .) Is derived.

  The imaging device 41 is a device that captures an image around the excavator, and outputs the captured image to the controller 30. In the present embodiment, the image pickup apparatus 41 is a wide camera that employs an image pickup device such as a CCD, and is mounted on the upper part of the upper swing body 3 such that the optical axis faces obliquely downward.

  The output device 50 is a device that outputs various types of information, and includes, for example, an in-vehicle display that displays various types of image information, an in-vehicle speaker that outputs various types of audio information. In the present embodiment, the output device 50 outputs various types of information in response to control commands from the controller 30.

  The object detection unit 31 is a functional element that detects an object existing around the excavator. In the present embodiment, the object detection unit 31 derives the actual position of the reflector based on the information on the irradiation direction and the reflector distance output by the scanning distance measuring device 40, and the actual position is within a predetermined distance range from the shovel. If there is, the reflector is detected as an object to be detected (object to be subjected to subsequent processing).

  In addition, the object detection unit 31 stores in advance the position of a predetermined object existing within the measurement range of the scanning distance measuring device 40, and excludes the predetermined object from the detection target. Note that the predetermined object existing within the measurement range of the scanning distance measuring device 40 is an object whose relative position with respect to the scanning distance measuring device 40 does not change even when the excavator moves. A turning mechanism 2 is included. Specifically, even when the scanning distance measuring device 40 derives the position of the reflector with respect to the turning mechanism 2, the object detection unit 31 does not detect the reflector as an object to be detected. Further, even when the object detection unit 31 derives the position of the reflector closer to the scanning distance measuring device 40 than the turning mechanism 2 in the direction in which the turning mechanism 2 is viewed from the scanning distance measuring device 40, The reflector is not detected as an object to be detected. This is because there is no object to be detected between the turning mechanism 2 and the scanning distance measuring device 40. For this reason, the piping or the like installed between the lower traveling body 1 and the upper swing body 3 is desirably disposed so as to pass through a monitoring unnecessary range that is a range between the swing mechanism 2 and the scanning distance measuring device 40. Is done. The scanning distance measuring device 40 may not irradiate the laser beam 40X in the direction in which the predetermined object exists. This is because there is no need for monitoring.

  FIG. 4 is a top view of the shovel showing the influence of the turning mechanism 2 excluded from the detection target as a predetermined object on the measurement range 40R of the scanning distance measuring device 40. The range indicated by the dotted circle represents the measurement range 40R of the scanning distance measuring device 40. A range 40Ra indicated by hatching indicates a blind spot range in which the laser beam 40X of the scanning distance measuring device 40 cannot reach due to the presence of the turning mechanism 2. A range 40Rb indicated by dot hatching represents a monitoring unnecessary range that does not need to be monitored due to the presence of the turning mechanism 2. 4 to 7, the illustration of the excavation attachment is omitted for the sake of clarity.

  Further, as shown in FIG. 4, the scanning distance measuring device 40 is attached at a position as far as possible from the pivot axis. This is because the blind spot range 40Ra decreases as the distance from the pivot axis increases.

  Further, the object detection unit 31 may determine whether the object is an obstacle that hinders measurement by the scanning distance measuring device 40 based on the detected position of the object. For example, when the actual position of the detected object is a position corresponding to the side wall of the upper swing body 3, the object detection unit 31 is an obstacle such as earth and sand that hinders measurement by the scanning distance measuring device 40. Is determined. In this case, the object detection unit 31 may output a control command to the output device 50 to notify the driver that there is an obstacle such as earth and sand on the side wall of the upper swing body 3.

  The human candidate extraction unit 32 is a functional element that extracts human candidates from the objects detected by the object detection unit 31. In the present embodiment, the person candidate extraction unit 32 extracts an object having a predetermined range of dimensions among the objects detected by the object detection unit 31 as a human candidate. For example, the object detection unit 31 determines the position of the human center based on the hypothesis of the weight value according to the distance and the human dimensions (body thickness, width, etc.) at each measurement point measured by the continuously irradiated laser beam 40X. Given distributions, superimpose the distributions with weights to find the probability distribution within the measurement range. Next, the maximum point of the obtained probability distribution is detected, and when the value of the probability distribution at the maximum point is larger than a predetermined threshold, the maximum point is specified as the position of the human candidate.

  The person identifying unit 33 is a functional element that identifies whether the person candidate extracted by the person candidate extracting unit 32 is a person. In the present embodiment, the person identification unit 33 corresponds to an image portion (hereinafter referred to as a “person candidate image portion”) corresponding to the person candidate extracted by the person candidate extraction unit 32 in the captured image captured by the imaging device 41. An image recognition process is performed to identify whether the candidate person is a person. Specifically, the person identification unit 33 captures an image using a correspondence relationship between coordinates in a three-dimensional space and coordinates in a captured image plane derived from the mounting position of the scanning distance measuring device 40 and the mounting position of the imaging device 41. The person candidate image portion in the image is specified. The human candidate image portion is an image portion having a predetermined size. Then, the human identification unit 33 uses a known image recognition process such as an image recognition process using an image feature description represented by a HOG (Histograms of Oriented Gradients) feature and a classifier generated by machine learning, as a human candidate. Identify if is a person. The rate at which the person identification unit 33 identifies a person candidate as a person increases as the person candidate extraction unit 32 extracts the person candidate with higher accuracy. Note that the person identifying unit 33 identifies that all of the human candidates are humans in a case where a captured image with a desired quality cannot be obtained in an environment that is not suitable for imaging such as at night or in bad weather. All of the human candidates extracted by the extraction unit 32 may be identified as people. This is to prevent human detection omissions.

  Next, a case where an obstacle M1 such as earth and sand exists between the lower traveling body 1 and the upper swing body 3 will be described with reference to FIG. FIG. 5 is a top view of the shovel showing the influence of the obstacle M1 such as earth and sand on the lower traveling body 1 on the measurement range 40R of the scanning distance measuring device 40, and corresponds to FIG. The range indicated by the dotted circle represents the measurement range 40R of the scanning distance measuring device 40. A range 40Ra indicated by hatching indicates a blind spot range in which the laser beam 40X of the scanning distance measuring device 40 cannot reach due to the presence of the turning mechanism 2. A range 40Rc indicated by hatching indicates a blind spot range where the laser beam 40X cannot reach due to the presence of the obstacle M1.

  As described above, the obstacle M1 between the lower traveling body 1 and the upper swing body 3 causes the blind spot range 40Rc within the measurement range 40R of the scanning distance measuring device 40, and the object detection capability of the object detection system 100 is increased. It will decrease.

  Therefore, the excavator on which the object detection system 100 according to the embodiment of the present invention is mounted desirably has the laser beam 40X of the scanning distance measuring device 40 passing through the gap between the lower traveling body 1 and the upper swing body 3. A structure that prevents the obstacle M1 from being obstructed (hereinafter referred to as “laser light protection structure”) is provided.

  FIG. 6 is a side view showing a configuration example of an excavator provided with the laser beam protection structure 60. FIG. 6A shows a state after the laser beam protection structure 60 is assembled, and FIG. The state before the laser beam protection structure 60 is assembled is shown.

  The laser beam protection structure 60 mainly includes a light guide plate 61 and a protection plate 62. The light guide plate 61 is a plate-like member that transmits the laser light generated by the scanning distance measuring device 40, and is formed of a light transmissive material such as acrylic resin. The protection plate 62 is a plate-like member that protects the scanning distance measuring device 40 and the light guide plate 61, and is formed of a highly rigid material such as metal, for example. In addition, each of the light guide plate 61 and the protection plate 62 has a size capable of covering the entire lower surface of the upper swing body 3, for example. However, each of the light guide plate 61 and the protection plate 62 is not necessarily required to cover a portion corresponding to the blind spot range where the laser beam 40X cannot reach due to the presence of the turning mechanism 2, a portion corresponding to the monitoring unnecessary range, and the like. Further, the lower surface of the upper swing body 3, the light guide plate 61, and the protection plate 62 are coupled to each other by an arbitrary method using an adhesive, a bolt, or the like. In the present embodiment, the scanning distance measuring device 40 is arranged so that the position of the portion that emits the laser light 40 </ b> X of the optical window portion 40 b corresponds to the position of the light guide plate 61. The main body 40a is embedded in the lower surface of the counterweight, and the lid 40c is embedded in the protective plate 62. Therefore, the laser light 40X passes through the inside of the light guide plate 61 and reaches the periphery of the shovel.

  With this configuration, the laser beam protection structure 60 can prevent the laser beam 40X from being blocked at the lower traveling body 1 by an obstacle M1 such as earth and sand (see FIG. 5). Therefore, it is possible to prevent the obstacle M1 such as earth and sand from expanding the blind spot range in the measurement range 40R of the scanning distance measuring device 40. Further, the laser beam protection structure 60 can prevent the scanning distance measuring device 40 from being destroyed by contact with an external object.

  Next, another laser light protection structure 70 will be described with reference to FIGS. FIG. 7 is a side view showing a configuration example of an excavator provided with another laser light protection structure 70, and FIG. 7A shows a state after the laser light protection structure 70 is assembled. (B) shows a state before the laser beam protection structure 70 is assembled. FIG. 8 is a top view of the shovel showing the influence of the laser beam protection structure 70 on the measurement range 40R of the scanning distance measuring device 40, and corresponds to FIGS.

  The laser beam protection structure 70 mainly includes a window material 71 and a protection plate 72. The window member 71 is a frame member that transmits laser light generated by the scanning distance measuring device 40, and is formed of a light-transmitting material such as acrylic resin. Moreover, the window material 71 forms a frame along the edge of the lower surface of the upper swing body 3 as shown by dot hatching in FIG. 8. Note that the window member 71 may be composed of one member or may be composed of a plurality of members.

  The protection plate 72 is a plate-like member that protects the scanning distance measuring device 40 and the window material 71, and is formed of, for example, a highly rigid material such as metal. Further, the protection plate 72 is coupled to the lower surface of the upper swing body 3 via post portions 72 a to 72 f (the post portions 72 d to 72 g are not visible in FIG. 7). It fixes so that the window material 71 may be pinched | interposed between them. The protective plate 72 has a size that can cover the entire lower surface of the upper swing body 3. However, each of the window member 71 and the protection plate 72 does not necessarily need to surround or cover a portion corresponding to a blind spot range where the laser beam 40X cannot reach due to the presence of the turning mechanism 2, a portion corresponding to a monitoring unnecessary range, and the like.

  Further, in the present embodiment, in the scanning distance measuring device 40, the position of the portion that emits the laser light 40 </ b> X of the optical window portion 40 b is surrounded by the window material 71 between the lower surface of the upper swing body 3 and the protection plate 72. It arrange | positions so that it may correspond to the position of the light guide space. The main body 40a is embedded in the lower surface of the counterweight, and the lid 40c is embedded in the protective plate 72. Therefore, the laser light 40X reaches the periphery of the shovel through the light guide space.

  With this configuration, the laser beam protection structure 70 can prevent the laser beam 40X from being blocked at the lower traveling body 1 by an obstacle M1 (see FIG. 5) such as earth and sand. Therefore, it is possible to prevent the obstacle M1 such as earth and sand from expanding the blind spot range in the measurement range 40R of the scanning distance measuring device 40. Further, the laser beam protection structure 70 can prevent the scanning distance measuring device 40 from being destroyed by contact with an external object.

  In the object detection system 100 mounted on an excavator provided with the laser beam protection structure 70, the object detection unit 31 includes post units 72a to 72g that are within the measurement range of the scanning distance measurement device 40, as shown in FIG. Are previously stored, and the post portions 72a to 72g are excluded from detection targets. A range 40Ra indicated by hatching in FIG. 8 represents a blind spot range where the laser beam 40X cannot reach due to the presence of the turning mechanism 2. Further, the five ranges 40Rd indicated by hatching indicate blind spots that the laser beam 40X cannot reach due to the presence of the post portions 72b to 72f. Note that a range 40Re indicated by hatching indicates a region where the laser beam 40X cannot reach due to the presence of the worker P.

  Specifically, the object detection unit 31 does not detect the reflector as the detection target object even when the scanning distance measuring device 40 derives the position of the reflector with respect to each of the post units 72b to 72f. . Further, the object detection unit 31 derives the position of the reflector closer to the scanning distance measuring device 40 from each of the post portions 72b to 72f in the respective existence directions of the post portions 72b to 72f as viewed from the scanning distance measuring device 40. Even if it is a case, the reflector is not detected as an object to be detected. This is because there is no object to be detected between each of the post portions 72b to 72f and the scanning distance measuring device 40. Since the post portions 72a and 72g are included in the blind spot range 40Ra, the reflector is not detected as an object to be detected.

  Further, it is desirable that the number of post portions is as small as possible, and it is desirable to install the post portions at a position as far as possible from the scanning distance measuring device 40. This is because the blind spot range 40Rd decreases as the number decreases and the distance from the scanning distance measuring device 40 increases.

  Further, when the scanning distance measuring device 40 derives the position of the reflector with respect to the worker P, the object detection unit 31 detects the reflector as an object to be detected. This is because the presence position of the worker P is not included in the position stored in advance.

  Further, as shown in FIG. 8, when the object detection unit 31 detects any object M2 at a position directly above the right lower traveling body 1 and adjacent to the window member 71, the object M2 is detected as a scanning distance. It is determined that the obstacle is obstructing measurement by the measurement device 40. This is because it can be determined from the position where the object M2 is present that the object M2 is not an object to be detected but earth or sand attached to the window member 71. In this case, the object detection unit 31 outputs a control command to the output device 50, displays on the in-vehicle display that there is a possibility that an obstacle such as earth and sand is attached to the window material 71, or makes a sound from the in-vehicle speaker. It may be output to notify the driver of the excavator to that effect.

  With the above configuration, the object detection system 100 is configured such that the laser beam of the scanning distance measuring device 40 passes between the lower traveling body 1 and the upper swinging body 3 so as to pass through the gap between the lower traveling body 1 and the upper swinging body 3. A scanning distance measuring device 40 is provided in the gap therebetween. Therefore, the entire blind spot area on the rear and side of the excavator can be monitored by the single scanning distance measuring device 40.

  Further, the object detection system 100 passes through the light guide plate 61 attached to the lower surface of the upper swing body 3 by the laser light of the scanning distance measuring device 40 or the window material 71 and the protection plate attached to the lower surface of the upper swing body 3. The scanning distance measuring device 40 is provided so as to pass through the light guide space formed by 72. Therefore, it is possible to prevent the laser light from being blocked by the earth and sand which are scraped up and accumulated on the lower traveling body 1.

  In addition, the object detection unit 31 excludes predetermined objects such as the turning mechanism 2 and the post units 72a to 72g existing within the measurement range of the scanning distance measuring device 40 from detection targets. Therefore, useless detection processing can be omitted, and the detection processing load can be reduced and the detection processing speeded up.

  Moreover, the person candidate extraction unit 32 extracts an object having a predetermined range of dimensions among the objects detected by the object detection unit 31 as a human candidate. Therefore, an object that is not suitable as a human candidate can be efficiently excluded from the human candidate.

  In addition, the person identifying unit 33 performs image recognition processing on an image portion corresponding to a person candidate in an image captured by the imaging device 41 to identify whether the person candidate is a person. Therefore, it is possible to perform image recognition processing only on image portions corresponding to a smaller number of human candidates than the number of objects detected by the object detection unit 31, thereby realizing a reduction in load related to identification processing, speeding up of identification processing, and the like. it can. Further, by performing detailed image recognition processing on a limited number of person candidate image portions, it is possible to identify with high accuracy whether or not the person candidate is a person.

  The object detection unit 31 determines whether the object is an obstacle that hinders measurement by the scanning distance measuring device 40 based on the detected position of the object. Therefore, it is possible to promptly notify the driver of the excavator that an obstacle such as earth and sand has adhered to the light guide plate 61 or the window material 71.

  Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

  For example, in the above-mentioned embodiment, the blind spot area of the excavator is covered using a single scanning distance measuring device 40, but the blind spot area of the shovel is covered using two or more scanning distance measuring apparatuses 40. Also good.

  In the above-described embodiment, the excavator blind spot area is imaged using three cameras, but the excavator blind spot area may be imaged using one, two, or four or more cameras.

  DESCRIPTION OF SYMBOLS 1 ... Lower traveling body 1A, 1B ... Hydraulic motor 2 ... Turning mechanism 3 ... Upper turning body 4 ... Boom 5 ... Arm 6 ... Bucket 7 ... Boom cylinder 8 ... arm cylinder 9 ... bucket cylinder 10 ... cabin 30 ... controller 31 ... object detection part 32 ... person candidate extraction part 33 ... person identification part 40 ... scanning distance Measuring device 40a ... Main body 40b ... Optical window 40c ... Lid 40R ... Measurement range 40Ra, 40Rc, 40Rd ... Blind spot range 40Rb ... Monitoring unnecessary range 40X ... Laser light DESCRIPTION OF SYMBOLS 41 ... Imaging device 50 ... Output device 60 ... Laser beam protection structure 61 ... Light guide plate 62 ... Protection plate 70 ... Laser beam protection structure 71 ... Window material 72 ... protection 72a to 72g ... post portion 100 ... object detection system M1 ... obstacle M2 ... object P ... worker

Claims (6)

An object detection system for a construction machine that detects an object existing around a construction machine including an upper turning body mounted on a lower traveling body via a turning mechanism,
A device for detecting an object attached to the lower side of the upper swing body separately from the camera mounted so that the optical axis is obliquely downward at the upper part of the upper swing body,
The device is a device for generating a laser so that an area measured by the device overlaps an area captured by the camera, and an optical axis of the laser is between the lower traveling body and the upper swing body. It is attached to the lower side of the upper swing body so that it can detect the object existing in the rear and the side by passing through the gap between them and reaching the rear and the side of the upper swing body, respectively. Yes,
Object detection system for construction machinery. The apparatus is an apparatus that detects an object using a laser generated by a laser generator, and has a main body that accommodates the laser generator and a light receiver. It is attached to the lower side of the upper swing body so that the distance between the object and the device in each of the rear and side can be detected.
The construction machine object detection system according to claim 1. The apparatus derives the actual position of the reflector based on the information on the laser irradiation direction and the reflector distance, and detects the reflector as an object if the actual position is within a predetermined distance range from the construction machine.
The construction machine object detection system according to claim 1 or 2. An imaging device that captures an image around the upper swing body;
A human detection unit that detects a person based on the output of the device and the output of the imaging device;
The construction machine object detection system according to any one of claims 1 to 3. The apparatus is attached to the lower surface of the counterweight so that the optical axis of the laser passes through the gap between the lower traveling body and the upper swing body,
The object detection system for construction machines according to any one of claims 1 to 4. The apparatus detects an object in an area of less than 360 degrees corresponding to a rear side, a left side, a right side, and a front side that is obstructed by a turning mechanism of the upper revolving body, including a rear side. ,
The construction machine object detection system according to any one of claims 1 to 5.

Images