JP6046286B1 - Image processing device - Google Patents

Abstract

An object of the present invention is to provide an image processing apparatus capable of suppressing a decrease in detection accuracy of a landing situation due to a shift in a camera mounting angle. According to an embodiment, there is provided an image processing apparatus that executes image processing on an image including a landing taken by a camera in order to detect a situation of a landing where a user gets on the car. The image processing apparatus according to the embodiment includes an acquisition unit, a calculation unit, and a correction unit. The acquisition unit acquires an image captured by the camera. The calculation means calculates a difference between the position of the reference portion and the reference position when the position of the reference portion included in the acquired image does not match the position that becomes a reference that is predetermined with respect to the reference portion. Is calculated. The correcting unit corrects the setting value related to the image processing based on the calculated difference. [Selection] Figure 4

Description

  Embodiments described herein relate generally to an image processing apparatus.

  Generally, a door (hereinafter referred to as a car door) is attached to a car on which a user rides in an elevator system.

  In such an elevator system, when a car arrives at the landing designated by the user (landing), the user can get on the car by opening the car door. Become.

  On the other hand, when the car moves up and down (up or down), the safety of the user can be ensured by closing the car door.

Japanese Patent No. 5476768

  Here, since the above-described car door is opened and closed using a motor, for example, a user may be caught by the car door when getting on.

  For this reason, for example, by detecting a landing situation (for example, a user who is going to get on the car) using a camera (image taken by the camera), an accident such as the user being caught by the door is prevented. It is possible.

  However, when the camera mounting angle (viewing angle, etc.) is deviated, there is a possibility that the detection accuracy of the landing situation is lowered.

  Therefore, the problem to be solved by the present invention is to provide an image processing apparatus capable of suppressing a decrease in detection accuracy of a landing situation due to a shift in the mounting angle of a camera.

According to the embodiment, there is provided an image processing apparatus that performs image processing on an image including the landing taken by a camera in order to detect a situation of the landing where the user gets on the car. The image processing apparatus according to the embodiment includes an acquisition unit, a calculation unit, and a correction unit. The acquisition means acquires an image photographed by the camera. The calculation means includes a position of the reference portion and a position serving as the reference when the position of the reference portion included in the acquired image does not match a position serving as a reference determined in advance with respect to the reference portion. The difference is calculated. The correction unit corrects a setting value related to the image processing based on the calculated difference. The set value related to the image processing includes a region where mask processing is performed in order to reduce the influence of disturbance light on the image processing. The area where the mask process is performed includes an area where a three-way frame provided at the entrance of the hall is located.

The figure which shows the structure of the elevator system containing the image processing apparatus which concerns on embodiment. The figure for demonstrating concretely the attachment angle of a camera. The figure which shows an example of the image image | photographed with a camera. FIG. 2 is a block diagram showing a functional configuration of the image processing apparatus. 6 is a flowchart illustrating a processing procedure of the image processing apparatus in a calibration function. The figure which shows an example of the image image | photographed with the said camera in the state in which the attachment angle of a camera has not shifted | deviated. The figure which shows an example of the image image | photographed with the said camera in the state which the attachment angle of the camera has shifted | deviated. The figure which shows an example of the mask area | region after correction | amendment.

  Hereinafter, embodiments will be described with reference to the drawings.

  FIG. 1 shows a configuration of an elevator system including an image processing apparatus according to the present embodiment. In the elevator system, a car 10 connected by a main rope and a counterweight (not shown) are provided in a hoistway and are respectively supported by guide rails (not shown) so as to be able to move up and down. Yes. A hoisting machine around which a main rope is wound is provided at the upper part of the hoistway. In the elevator system, the car 10 (and the counterweight) can be moved up and down by driving such a hoisting machine.

  An openable / closable car door 10 a is installed at the entrance of the front of the car 10. The opening / closing control of the car door 10a is performed by, for example, a car control device (not shown) installed in the upper part of the car 10.

  According to such an elevator system, when the car 10 arrives at the landing 20 designated by the user (landing), the car door 10a is opened, so that the user can get on the car 10. Can be made. Further, by closing the car door 10a during the raising / lowering operation of the car 10, it is possible to ensure the safety of the user who is on the car 10.

  Although not shown in FIG. 1, a landing door is provided at the landing 20 in order to prevent a user who stands by at the landing 20 to get on the car 10 from falling on the hoistway. It has been. This landing door opens and closes with the opening and closing of the car door 10a.

  In the present embodiment, a camera (imaging device) 30 that captures an image including a landing 20 where a user gets on the car 10 is installed in the car 10. Specifically, the camera 30 is installed in the vicinity of the center of the curtain plate portion provided above the entrance / exit of the car 10, and the car 30 is installed in the car 10 as shown in FIG. An image including a boundary portion between the car 10 and the landing 20 when the landing door provided at the door 10a and the landing 20 is in an open state is taken. In other words, the camera 30 is installed at a position where the area on the landing 20 side and the area on the car 10 side can be photographed. Assume that the camera 30 can capture several frames of images per second, such as an in-vehicle camera.

  Here, the attachment angle (position) of the camera 30 will be specifically described with reference to FIG. In the present embodiment, the camera 30 is installed at a height h (for example, about 2,100 mm) from the floor surface of the car 10, and captures a range of L1 + L2. L1 is a shooting range on the landing 20 side, and is a range of, for example, about 2,000 to 3,000 mm from the car door 10a (or the landing door). L2 is an imaging range on the car 10 side, and is assumed to be, for example, a range of about 350 to 500 mm from the car door 10a toward the back of the car. L1 and L2 are ranges in the depth direction, and the range in the width direction (direction orthogonal to the depth direction) is at least larger than the lateral width of the car 10.

  In the camera 30, the angle of view (α) is set in advance so that the above-described shooting range (area) can be shot.

  FIG. 3 shows an example of an image taken by the camera 30 in the present embodiment. According to the mounting angle of the camera 30 described in FIG. 2, the camera 30 can capture the image 100 when the car door 10 a (and the landing door) is in the open state. Specifically, the image 100 includes an area 101 on the landing 20 side and an area 102 on the car 10 side as shown in FIG.

  Although not shown in FIG. 1 described above, a sill (sill) for guiding the opening and closing of the car door is provided on the car 10 side. Similarly, a sill (sill) for guiding the opening and closing of the landing door is provided on the landing 20 side. For this reason, in the boundary part of the area 101 on the landing 20 side and the area 102 on the car 10 side in the image 100 shown in FIG. 1 includes a sill (car side sill) 112 provided in the vehicle.

  In addition, the entrance / exit of the hall 20 is provided with a frame of the entrance / exit called a three-way frame. The three-way frame is composed of a vertical frame on both sides of the entrance / exit and an upper horizontal frame. According to the mounting angle of the camera 30 described above, the image 100 further includes a three-sided frame (vertical frames on both sides) 113. Since the camera 30 is provided below the upper frame of the three-way frame 113 (the curtain plate portion), the image 100 does not include the upper frame.

  When there is a user who wants to get on the car 10 at the landing 20, as shown in FIG. 3, the area 200 on the landing 20 side in the image 100 includes the user 200.

  Returning to FIG. 1 again, an image processing device 40 is connected to the camera 30. The image processing device 40 executes image processing (analysis processing) on an image (video) taken by the camera 30 in real time. The image processing apparatus 40 is provided independently of the camera 30, for example. Further, the image processing device 40 may be incorporated in the above-described car control device or the like.

  The image processing device 40 has a function of detecting the status of the hall 20 by executing image processing on an image captured by the camera 30. Specifically, the image processing apparatus 40 can detect, for example, the presence or absence of the user 200 who is going to get on the car 10 from the hall 20 based on the area 101 on the hall 20 side in the image 100. According to this, when it is detected by the linked operation of the image processing device 40 and the car control device that there is a user 200 trying to get on the car 10, for example, to the car 10 of the user 200. The car door 10a can be controlled to be kept open until the boarding of the car is completed.

  In addition, the image processing apparatus 40, for example, based on the area 102 on the car 10 side in the image 100, the user's hand or the like who is on the car 10 during the door opening operation is drawn into the car door 10a (door bag). It is also possible to detect such a situation (that is, the situation in the car 10). According to this, when a situation in which the user's hand is likely to be drawn into the car door 10a is detected by the linkage operation of the image processing device 40 and the car control device, for example, the control to interrupt the door opening operation, for example. Is possible.

  Here, as described above, the image captured by the camera 30 includes a three-sided frame, but the three-sided frame often has a relatively high light reflectance as compared with other parts in the image. . For this reason, when an image is captured by the camera 30, the intensity of light reflected by the three-way frame may be strong, but this light affects the image processing executed by the image processing device 40 described above as disturbance light. May give. Specifically, there may be a case where the user cannot be detected even though there is a user who tries to get on the car 10 from the landing 20. That is, the light reflected by the three-way frame may reduce the detection accuracy of the situation of the hall 20.

  Therefore, in the image processing apparatus 40, it is assumed that an area to be subjected to mask processing (an area including a three-sided frame) is set as a setting value related to image processing. According to this, the light reflected by the three-way frame that becomes disturbance light can be blocked by the mask process, so that the influence of the disturbance light can be reduced and the detection accuracy of the situation of the hall 20 can be improved. it can.

  Note that the area to be subjected to the mask process is set in advance based on a predetermined attachment angle of the camera 30.

  However, the camera 30 may not be attached at a predetermined attachment angle. Specifically, since the camera 30 is attached manually by an operator, there is a possibility that the camera 30 is attached in a state where the camera 30 is inclined by mistake. In such a case, a deviation occurs between the area where the mask processing is performed and the area including the three-way frame in the image actually captured by the camera 30. According to this, since the area including the three-way frame in the captured image is not appropriately masked, the detection accuracy of the situation of the hall 20 is affected by the influence of disturbance light (that is, light reflected by the three-way frame). Incurs a decline.

  Furthermore, since the camera 30 is installed above the entrance of the car 10 as described above, it may be similarly affected by ambient light even when the car 10 is inclined.

  Therefore, the image processing apparatus 40 according to the present embodiment has a setting value (described above) related to image processing when the mounting angle of the camera 30 is shifted (that is, the camera 30 is mounted in an inclined state). It has a function of correcting (automatic adjustment) (hereinafter referred to as a calibration function).

  FIG. 4 is a block diagram illustrating a functional configuration of the image processing apparatus 40 according to the present embodiment. Here, the functional configuration related to the calibration function described above will be mainly described.

  As illustrated in FIG. 4, the image processing device 40 includes a storage unit 41, an image acquisition unit 42, a reference part extraction unit 43, a difference calculation unit 44, a correction processing unit 45, and a notification processing unit 46.

  The storage unit 41 stores setting values related to the image processing described above. The set value relating to the image processing includes an area (coordinate value) where the three-way frame where the mask process is performed should be located. The storage unit 41 stores in advance a reference position (hereinafter referred to as a reference position) that is predetermined with respect to a reference portion to be described later. The reference position is the position of the reference part in the captured image when the camera 30 is accurately attached at a predetermined attachment angle.

  The image acquisition unit 42 acquires an image captured by the camera 30 (hereinafter referred to as a captured image).

  The reference part extraction unit 43 extracts a reference part included in the captured image acquired by the image acquisition unit 42. The reference part is a part serving as a reference for determining whether or not the mounting angle of the camera 30 described above is shifted, and includes, for example, a landing side sill and a car side sill. Note that the landing-side sill and the car-side sill are often formed of, for example, aluminum or stainless steel, and thus can be easily extracted from the captured image as compared with other parts.

  If the position of the reference part (position in the captured image) extracted by the reference part extraction unit 43 and the reference position stored in the storage unit 41 do not match, the difference calculation unit 44 The difference from the reference position is calculated. Note that the difference calculated by the difference calculation unit 44 is represented by, for example, coordinate values.

  The correction processing unit 45 acquires setting values related to image processing stored in the storage unit 41. The correction processing unit 45 corrects the set value (for example, an area where mask processing is performed) related to the acquired image processing based on the difference calculated by the difference calculation unit 44. The correction processing unit 45 stores the set value related to the corrected image processing in the storage unit 41. The settings relating to the corrected image processing are used in the image processing executed when the situation of the hall 20 is detected.

  Here, in the present embodiment, by correcting the setting value related to the image processing by the correction processing unit 45, the detection accuracy of the situation of the hall 20 is reduced even when the mounting angle of the camera 30 is shifted. However, during the operation of the elevator system, for example, the mounting angle of the camera 30 may be further shifted due to an impact on the car 10 or the camera 30 (that is, the camera 30 tilts). .

  In such a case, the notification processing unit 46 notifies (reports) to the monitoring center (the manager) who monitors the operation state of the elevator system, for example, that the mounting angle of the camera 30 has shifted. The difference in the mounting angle of the camera 30 means that the difference calculated again by the difference calculation unit 44 during the operation of the elevator system (that is, after the setting value related to image processing is corrected) is the setting related to the image processing described above. Whether the difference is larger than the difference calculated by the difference calculation unit 44 when correcting the value can be determined.

  Next, a processing procedure of the image processing apparatus 40 in the calibration function according to the present embodiment will be described with reference to the flowchart of FIG. Note that the processing shown in FIG. 5 is executed before the operation of the elevator system, for example.

  First, the image acquisition unit 42 acquires an image (captured image) captured by the camera 30 (step S1).

  The reference part extraction unit 43 extracts, for example, a landing-side sill and a car-side sill (region) from the photographed image as reference parts in the photographed image acquired in step S1 (step S2).

  Next, the difference calculation unit 44 acquires a reference position stored in the storage unit 41. The difference calculation unit 44 determines whether the mounting angle of the camera 30 is shifted based on the acquired reference position (step S3). In this case, the difference calculation unit 44 executes the determination process based on whether or not the position in the captured image of the landing-side sill and the car-side sill (reference part) extracted in step S2 matches the reference position. To do. Specifically, when the positions in the captured images of the landing-side sill and the car-side sill coincide with the reference position, it can be determined that the mounting angle of the camera 30 is not shifted. On the other hand, when the positions in the captured images of the landing-side sill and the car-side sill do not match the reference position, it can be determined that the mounting angle of the camera 30 is shifted.

  If it is determined that the mounting angle of the camera 30 is not shifted (NO in step S3), it is not necessary to correct the setting value related to image processing, and thus the processing ends.

  On the other hand, when it is determined that the mounting angle of the camera 30 is shifted (YES in step S3), the difference calculation unit 44 determines the position of the landing side sill and the car side sill (reference part) in the captured image and the reference position. Is calculated (step S4).

  Next, the correction processing unit 45 acquires a setting value related to image processing stored in the storage unit 41. The correction processing unit 45 corrects the area to be subjected to the mask process (area where the three-sided frame should be located) included in the acquired setting value related to image processing based on the difference calculated in step S4 (step S5). ). In the following description, an area on which mask processing included in a setting value related to image processing is performed is referred to as a mask area for convenience.

  Here, with reference to FIG.6 and FIG.7, step S5 is demonstrated concretely. In FIG. 6 and FIG. 7, the same reference numerals as those in FIG. Similarly, similar parts in FIGS. 6 and 7 will be described with the same reference numerals.

  First, FIG. 6 shows an example of an image photographed by the camera 30 in a state where the mounting angle of the camera 30 is not shifted (the camera 30 is not tilted).

  A reference position 400 shown in FIG. 6 represents the position of the reference portion (the landing-side sill 111 and the car-side sill 112) when the camera 30 is accurately attached at a predetermined attachment angle.

  In the image 310 shown in FIG. 6, since the positions of the landing-side sill 111 and the car-side sill 112 (boundary portions thereof) coincide with the reference position 400, it is determined that the mounting angle of the camera 30 is not shifted. be able to.

  Here, the area of the three-way frame 113 in the image 310 matches the mask area 311 as shown in FIG. That is, when the mounting angle of the camera 30 is not shifted, the mask process can be appropriately performed on the area of the three-way frame 113.

  On the other hand, FIG. 7 shows an example of an image photographed by the camera 30 in a state where the mounting angle of the camera 30 is shifted (the camera 30 is tilted).

  In the image 320 shown in FIG. 7, since the positions of the landing side sill 111 and the car side sill 112 (boundary portions thereof) and the reference position 400 do not match, it is determined that the mounting angle of the camera 30 is shifted. be able to.

  Here, the region of the three-way frame 113 in the image 320 does not coincide with the mask region 311 as shown in FIG. That is, the mask region 311 is set in a region where the three-way frame 113 is positioned when the camera 30 is accurately attached at a predetermined attachment angle, and the camera 30 is attached at a different angle. In the image 320, the area where the three-way frame 113 is located is shifted from the position of the mask area 311. That is, when the mounting angle of the camera 30 is shifted, the light reflected by the portion of the three-way frame 113 that does not overlap with the mask region 311 affects as disturbance light, and the detection accuracy of the situation of the hall 20 is improved as described above. There is a possibility of lowering.

  For this reason, in step S5, the correction processing unit 45 determines the image 320 based on the difference calculated in step S4 (that is, coordinate values representing the inclination of the landing-side sill 111 and the car-side sill 112 with respect to the reference position). The position of the mask region 311 is corrected so as to coincide with the region of the inner three-way frame 113. In this case, the correction processing unit 45 may correct the mask region 311 so as to deform it.

  Returning to FIG. 5 again, the mask area corrected in step S5 (mask area after correction) is stored in the storage unit 41 (step S6).

  Even if the mounting angle of the camera 30 is shifted by using such a corrected mask region, for example, an image taken by the camera 30 during operation of the elevator system as shown in FIG. Mask processing can be appropriately performed on the region of the three-way frame 113 in 330.

  In addition, in FIG. 5, although demonstrated as what the reference | standard site | part is a landing side sill and a car side sill, one of the said landing side sill and the car side sill may be sufficient. In addition, the reference part may be a part other than the landing-side sill and the car-side sill as long as it can be extracted from the captured image.

  Further, in step S3 shown in FIG. 5, it is determined whether or not the mounting angle of the camera 30 is deviated depending on whether or not the positions of the landing side sill and the car side sill in the photographed image coincide with the reference position. As described above, even when the mounting angle of the camera 30 is deviated, if the deviation (amount) does not affect the detection accuracy of the situation of the landing 20, the image processing is performed. A configuration in which the set value (mask region) relating to the correction is not corrected may be employed. That is, the process of step S3 is executed based on whether or not the difference between the position in the captured image of the landing side sill and the car side sill and the reference position is within a predetermined range, and the difference is determined in advance. If it is not within the specified range, it may be determined that the mounting angle of the camera 30 is shifted.

  As described above, in the present embodiment, an image (captured image) captured by the camera 30 is acquired, and the position of the reference portion and the reference position (predetermined reference position) included in the acquired captured image. ) And a setting value related to image processing is corrected based on the calculated difference. The set value relating to the image processing includes, for example, a region where mask processing is performed in order to reduce the influence of disturbance light on the image processing. Moreover, the area | region where a mask process is performed contains the area | region in which the three-way frame provided in the entrance / exit of the hall 20, for example is located.

  In the present embodiment, with such a configuration, for example, when the camera 30 attached by an operator is inclined or the car 10 itself is inclined, the installation angle of the camera 30 (that is, the operation angle of the camera 30 before the elevator system is operated) Even when the shooting angle of the landing 20 or the like is shifted, it is possible to appropriately perform mask processing on the area of the three-way frame. It is possible to suppress a decrease in detection accuracy of the situation (or in the car 10).

  In the present embodiment, the captured image includes a landing door provided in the landing 20 and a boundary portion between the landing 20 and the car 10 when the car door 10a provided in the car 10 is in an open state. The reference portion includes at least one of a landing-side sill and a car-side sill. In this embodiment, because of such a configuration, the landing-side sill and the car-side sill can be extracted from the captured image relatively easily, so the position of the reference portion is accurately compared with the reference position. Thus, it is possible to detect (determine) a shift in the mounting angle of the camera 30.

  Further, in the present embodiment, since the camera 30 is provided in the curtain plate part above the entrance of the car, both the landing 20 and the car 10 can be set as the shooting range. Thus, the situation of the landing 20 (whether there is a user trying to get on the car 10 from the landing 20) and the situation inside the car 10 (the situation where the user's hand is likely to be drawn into the car door 10a), etc. are detected. It becomes possible to do.

  Further, in the present embodiment, after the setting value related to image processing is corrected, the difference between the position of the reference part and the reference position included in the image photographed by the camera 30 is further calculated, and the further calculated difference is the image. If the difference is larger than the difference calculated when the setting value related to the process is corrected, the abnormality is notified to the administrator (that is, the mounting angle of the camera 30 is further shifted). In the present embodiment, with such a configuration, for example, after a setting value related to image processing is corrected, if a situation occurs in which the mounting angle of the camera 30 further shifts, management is performed so that the shift is corrected. Can be encouraged.

  In the present embodiment, the mask area (area where the mask process is performed) is corrected as the setting value related to the image processing. However, even if the setting value related to the image processing other than the mask area is corrected, Good.

  Specifically, in the image processing apparatus 40 according to the present embodiment, for example, when the car door 10a is in an open state, the exposure adjustment of the camera 30 is targeted for the area on the landing 20 side in order to detect the situation of the landing 20 When the car door 10a is in the closed state or when the door is being opened, the exposure adjustment of the camera 30 is performed for the area on the car 10 side in order to detect the situation inside the car 10 And In this case, the area on the landing 20 side and the area (spot) on the car 10 side that are subject to exposure adjustment are set in advance as set values relating to image processing, but the mounting angle of the camera 30 is shifted as described above. In such a case, the set area may deviate from the area on the landing 20 side or the area on the car 10 side where exposure adjustment should be performed, and appropriate exposure adjustment may not be performed. For this reason, in the present embodiment, a configuration may be adopted in which a region to be subjected to exposure adjustment when an image is captured by the camera 30 is corrected based on the difference between the position of the reference portion and the reference position.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 10 ... Car 10a ... Car door, 20 ... Platform, 30 ... Camera, 40 ... Image processing apparatus, 41 ... Storage part, 42 ... Image acquisition part, 43 ... Reference | standard part extraction part, 44 ... Difference calculation part, 45 ... Correction processing unit, 46 ... notification processing unit, 111 ... landing side sill, 112 ... car side sill, 113 ... three-way frame.

Claims (5)

In an image processing apparatus for executing image processing on an image including the landing taken by a camera in order to detect a situation of the landing where the user gets on the car,
Obtaining means for obtaining an image taken by the camera;
If the position of the reference part included in the acquired image does not match the reference position predetermined for the reference part, the difference between the reference part position and the reference position is calculated. A calculation means;
Correction means for correcting a set value related to the image processing based on the calculated difference, and
The set value related to the image processing includes a region where mask processing is performed in order to reduce the influence of disturbance light on the image processing,
The area where the mask process is performed includes an area where a three-way frame provided at the entrance of the hall is located.
An image processing apparatus. In an image processing apparatus for executing image processing on an image including the landing taken by a camera in order to detect a situation of the landing where the user gets on the car,
Obtaining means for obtaining an image taken by the camera;
If the position of the reference part included in the acquired image does not match the reference position predetermined for the reference part, the difference between the reference part position and the reference position is calculated. A calculation means;
Correction means for correcting a set value related to the image processing based on the calculated difference;
Comprising
The setting value related to the image processing includes an area to be subjected to exposure adjustment when an image is taken by the camera.
An image processing apparatus. In an image processing apparatus for executing image processing on an image including the landing taken by a camera in order to detect a situation of the landing where the user gets on the car,
Obtaining means for obtaining an image taken by the camera;
If the position of the reference part included in the acquired image does not match the reference position predetermined for the reference part, the difference between the reference part position and the reference position is calculated. A calculation means;
Correction means for correcting a set value related to the image processing based on the calculated difference;
Notification means;
Comprising
The calculation means further calculates a difference between a position of a reference portion included in the image acquired after correcting the setting value related to the image processing and a position serving as a predetermined reference,
The notification means, it said further if the calculated difference is greater than the calculated difference, images processor you characterized by notifying the abnormality to the administrator. The image taken by the camera includes a landing door provided at the landing and a boundary portion between the landing and the car when the car door installed in the car is in an open state.
The reference portion includes at least one of a landing-side sill provided for guiding opening / closing of the landing door and a passenger-side sill provided for guiding opening / closing of the car door. The image processing apparatus according to claim 1 , wherein the image processing apparatus is characterized. The image processing apparatus according to claim 4 , wherein the camera is installed above an entrance of the car.

Images